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1. EXECUTIVE SUMMARY 


This proposal describes a decision support project collaboration that includes the Kempe Center 
for the Prevention and Treatment of Child Abuse and Neglect (Kempe) and its partners, Chapin 
Hall at the University of Chicago (Chapin Hall), Intelligent Outcome Analytics (10A), and 
Pierson Computing Connections. Key staff include our CO-PIs; John Fluke, with the Kempe 
Center who will also serve as the main point of contact, Fred Wulczyn with Chapin Hall, and 
David Schwartz with lOA. In addition we will subcontract with Pierson Computing Connections, 
a Pennsylvania Department of General Services certified woman business enterprise (WBE), for 
technical purchase and support. Kempe staff are known for their groundbreaking research using 
an ecological approach to understanding factors affecting child welfare staff decision making and 
its relationship to the assessment and prediction of child welfare outcomes. The scope of Chapin 
Hall’s research and the variety of contexts in which that work is conducted leave Chapin Hall 
staff with a firm appreciation for the challenges inherent in developing, implementing and 
evaluating programs designed to enhance the functioning of child welfare systems with the 
ultimate goal of improving safety, permanency, and well-being outcomes for children and 
families. Kempe, Chapin, and lOA staff all have substantial experience using a variety of 
administrative data systems. 

A major and extremely complex challenge for decision making in social services is to determine 
how the needs of high risk populations can be optimally served to maximize outcomes at the 
system level. In other words, how can participants be meaningfully classified with regard to 
needs, and connected to services that are most likely to be effective? 

Our solution is to develop a decision support approach consisting of three integrated 
components: computerized adaptive testing/assessment (CAT-MH), intelligent outcome 
management and assessment (predictive modeling), and agent based simulation. The conceptual 
approach builds on technologies that the project team has developed and utilized individually in 
other settings. The concept is that data from the CAT-MH will be combined with other 
administrative data including other assessments where they exist to support the SPSS Modeler, 
these data in turn will help to define need clusters where the assessment and administrative data 
regarding service delivery for the clusters can be used to define baseline parameters for agent 
based simulation and resource allocation for evidence based or supported interventions. The 
intent is to develop a detailed design and test a proof of concept for two high-risk populations; 
adolescents in or at risk of foster care, and elders who are psychiatrically hospitalized or at risk 
of hospitalization. A focus of the project planning and evaluation process is to determine how the 
three tools can be integrated into the current DHS IT infrastructure including COGNOS. 

Our collaboration is proposing to conduct the first year design and proof and concept process 
described in the proposal for $ $756,743. 
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2. PROPOSAL NARRATIVE 


Description of Organizations 

This section describes a group of organizations and individuals coming together as a 
collaboration and describes our organizational capacity, experience, and compatibility with 
respect to a proposal for a successful design and proof of concept of data systems that support 
social service decision-making. The collaboration includes the Kempe Center for the Prevention 
and Treatment of Child Abuse and Neglect (Kempe) and its partners, Chapin Hall at the 
University of Chicago (Chapin Hall), Intelligent Outcome Analytics (10 A), and Pierson 
Computing Connections. It also describes our collective substantive experience in development, 
implementation, evaluation, and research in social services. The Kempe Center is the lead 
organization on this proposal and will be the point of contact for the County. That said, the 
organizations and consultants on our team constitute a collaborative approach. Each team 
member will makes a unique contribution to the proposed project, and leverages many years of 
experience in the development of decision support. In turn the strengths, compatibility, and 
contributions of each of these partners in our collaborative makes us uniquely suited to propose 
this work. 

1) Organizations’ history and experience conducting research, translating research into 
practice, developing/validating tools huilt on research, developing and implementing 
predictive models. 

The Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 
For 40 years, the Kempe Center has promoted understanding, knowledge, and best practices to 
prevent and treat child abuse and neglect locally, nationally, and internationally. Kempe fulfills 
its mission through research and evaluation; training, technical assistance, and education; and 
service delivery, advocacy, and consultation. Kempe is administratively located within the 
Department of Pediatrics at the University of Colorado Anschutz Medical Campus in Aurora, 
Colorado. Kempe’s 80 faculty and staff are experienced in designing, implementing, evaluating, 
and disseminating a wide range of evidence-based programs and initiatives (e.g., differential 
response, trauma-focused mental health treatment, family group decision making, risk and safety 
assessment, and community-based child abuse prevention) to prevent and treat child abuse and 
neglect as well as improve the health and well-being of children and families. When several staff 
members from the American Humane Association joined the faculty of Kempe in 2012, their 
addition made Kempe distinctive among similar organizations in the United States and 
internationally by increasing its breadth of expertise related to child welfare practice, policy, and 
research. 

The expertise and accomplishments of Kempe’s faculty reflect a deep commitment to conducting 
research and translating it into practice. With respect to the work proposed herein, Kempe staff 
are known for their groundbreaking research using an ecological approach to understanding 
factors affecting child welfare staff decision making and outcomes. This experience includes 1) 
the identification of case factors that influence the likelihood that child maltreatment will recur 
or that families and/or children will be the subject of repeat referral or re-reports; 2) the 
identification of agency, worker, and case characteristics’ that influence decision making 
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practices, 3) the impact of policies, practiees, and demographies on states’ child maltreatment 
reeurrenee rates, and 4) using research findings to develop safety and risk assessment tools, 
training, and polieies. These efforts have yielded tools and models that prediet variations in 
outcomes from the individual to the aggregate level. Moreover, the development and applieation 
of these and other analyses have involved the speeification of data extracts from child welfare, 
mental health, heath, benefits and other information systems in many states, construetion of 
event data, and the use of a range of predietive analytic procedures. Kempe staff are well-versed 
in a variety of methodological approaches and have lead studies involving randomized control 
trials, propensity-score comparisons and other quasi-experimental designs. Other speeific 
program and system evaluations eonducted by Kempe staff inelude the impaet of differential 
response, family group deeision making, and factors influencing child welfare disparities. 

Moreover, members of the Kempe Center researeh team are particularly skilled at the use of 
administrative data and utilizing the resourees of multiple data sourees for the assessment and 
predietion of ehild welfare outeomes. Kempe staff has substantial experienee using a variety of 
administrative data systems including State SACWIS systems and the federal NCANDS and 
AFCARS data sets; and 2) Foster Care Redesign for the Texas Department of Family and 
Protective Services (2010-), in whieh Kempe and Chapin Hall collaborated to use the Texas 
SACWIS system to support modifieations to the financial system for reimbursing out-of-home 
service providers. The depth and breadth of their experienees refieets a firm eommitment to the 
use of information teehnology to inform policy and practice decisions, advance the 
understanding of workload, develop models of ageney finanee and eosts, improve outcomes, 
edueate, strengthen, and transform agencies. 

Chapin Hall at the University of Chicago 

For more than 25 years Chapin Hall has foeused on building knowledge that improves policies 
and programs for ehildren and youth, families, and eommunities. The seope of Chapin Hall’s 
research and the variety of eontexts in whieh that work is eonducted leave Chapin Hall staff with 
a firm appreeiation for the challenges inherent in developing, implementing and evaluating 
programs designed to enhanee the functioning of child welfare systems with the ultimate goal of 
improving safety, permaneney, and well-being outcomes for ehildren and families. 

Chapin Hall is a nonpartisan poliey researeh center that provides researeh and teehnieal 
assistance services to federal, state, and munieipal governments as well as philanthropic 
foundations on a wide speetrum of issues ineluding performance-based contraeting, serviee 
system use, and measurement of ehild well-being. Chapin Hall also examines, evaluates, and 
doeuments initiatives that seek to support youth development and build the capaeity of 
neighborhoods, community organizations, schools, and families to care for their ehildren. 

Chapin Hall is a national leader in the development of knowledge about the experiences of 
ehildren in foster eare. Approximately 20 years ago, Chapin Hall developed the Multi-State 
Foster Care Data Arehive (FCDA), a repository of administrative data provided by ehild welfare 
ageneies in more than twenty states. The Arehive ineludes the foster care placement histories of 
nearly three million foster children. With the reeent additions of Texas and California, the data 
inelude about 70 pereent of the nation’s foster care population. Due to the unique methods for 
storing individual level information, eonsiderable analytie eapacity is embedded in the data 
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archive. The flexible file strueture permits a wide range of questions, lends itself to a variety of 
researeh methods, and when appropriate, can easily expand to incorporate information aeross 
domains. To facilitate aeeess to the data, researehers have developed interactive tools to extract 
data for the purpose of data mining. The tools are expressly designed to distribute aeeess to the 
data to a broad cross-seetion of child welfare professionals operating inside the ehild welfare 
system. This Data Arehive forms the eore resource of the Center for State Foster Care and 
Adoption Data (State Data Center), a partnership of the Ameriean Publie Human Serviees 
Association (APHSA) and Chapin Hall, which was established to bring member ehild welfare 
ageneies cutting-edge information teehnology for performance measurement. 

Chapin Hall staff are also accomplished at performing outeome evaluations for human serviee 
ageneies using eomplex and multifaeeted administrative data. The bulk of Chapin Hall’s work 
with public child welfare systems is done through Chapin Hall’s Center for State Child Welfare 
Data, described above. Using the FCDA, Chapin Hall staff have studied, among other topies, 
ehild welfare outeomes pertaining to raeial disparity in the ehild welfare system, the experienee 
of infants in foster care, the eireumstanees of youth aging out of foster care, and the intersection 
of child welfare with other ehild- and family-serving systems sueh as the eourts and the 
edueation, health, and mental health systems.^ Additionally, FCDA data are uploaded to a user- 
friendly, web tool (also developed by Chapin Hall researchers) that enables staff from 
partieipating state ageneies to answer their own business questions about systemic trends and 
outeomes for children in care using the agencies’ own administrative reeords.^ 

Among Chapin’s most reeent aceomplishments. State Data Center staff reeently completed a 
comprehensive link between foster care and Medicaid data. The link is at the individual, elaim, 
and plaeement level. The data aeeount for $2.6 billion worth of Medieaid elaims, which are 
retrievable on a person, plaoement/faeility-speeifie basis. 

Intelligent Outcome Analytics (lOA), LLC 

lOA is a Pennsylvania-based firm formed to help key deeision-makers rigorously build and 
deploy analytie tools and predietive models that ean be plugged into dashboards, decision 
support tools, and data warehouses. 10A leverages teams of interdisciplinary subject matter 
experts to support the proeess thoughtfully and cost-effectively. 

lOA’s founder and CEO, David Schwartz, has led several predietive analytie modeling 
engagements, ineluding projects for the New York Office of Children and Family Serviees, Tufts 
Healthplan Foundation, the Gap Foundation, etc. Consultants and staff from 10A are 
experienced in developing and deploying predietive analyties. These include applieations in 
Juvenile Justice and Child Protection. 

Pierson Computing Connection 

Pierson Computing Connection is a strong and experienced team able to perform teehnology 
implementation; software and hardware. The PCCi team provides signifieant experienee, a 
trustworthy team and a commitment to customer satisfaction. PCCi has more than twenty years 
experience in performing large seale deployments of IT hardware and software and is located in 


* For more information on this research program, see https://fcda.chapinhall.org/building-the-knowledge-base/ 

^ For more information about this technical assistance program, see https://fcda.chapinhall.org/knowledge-in-action/ . 
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Mechanicsburg, PA. During the last ten years PCCi has installed more than 130,000 PCs for 
customers throughout the east coast, including more than 30,000 within Pennsylvania. PCCi is a 
designated IBM software vendor for the IBM SPSS brand. 

The PCCi project management team has combined experience of more than forty years of 
experience managing local projects. Project Executive Debra Pierson, and Project Manager 
Janese Shenk, lead a team that works closely together to meet client needs with excellence. They 
are supported by Kristen Mitchell, PMP, and by our administrative team; experienced staff 
project managers who assist with every level of project detail. PCCi is a Pennsylvania 
Department of General Services certified woman business enterprise (WBE) and headquartered 
in a designated enterprise zone, PCCi specializes in providing hardware and software, project 
management and deployment services. 

2) Relevant experience in working with human services practitioners. 

The staff from the Kempe Center, Chapin Hall, 10A, and PCCi slated to work on this initiative 
have a well-established track record of working effectively with human service practitioners 
serving a variety of roles in supporting the development, implementation, and assessment of 
human service provision. Staff for this project have worked with human services practitioners at 
all levels, in virtually every state in the U.S., and in numerous international contexts. Staff from 
each of these entities are quite familiar with processes and pertinent questions related to 
engaging government analysts in identifying administrative data resources and existing sources 
of relevant information, obtaining relevant input and feedback from executive to front line staff, 
and understanding the specific organizational culture that underlies the context for a work 
project. 

More specifically, in addition to the EGC work described above, the Kempe Center has just 
completed a five year, federally funded project evaluating the implementation of differential 
response in child welfare systems in Ohio, Illinois and Colorado. Kempe staff also have a well- 
established and ongoing working relationship with the Colorado Department of Human Services 
and is in the midst of developing an evaluation method for assessing the delivery and efficacy of 
services to families screened out of the child welfare system. Kempe currently works with the 
Utah Department of Social Services as a member of the Title IV-e waiver evaluation team 
headed by the University of Utah’s College of Social Work Social Research Center (SRC). 
Kempe has also supported efforts of Earimer County Social Service in Colorado, the Texas 
Department of Eamily and Protective Services, and the South Dakota Department Social 
Services in the development of an evaluation framework, monitoring system, and assessment of 
their family group decision making programs. 

Kempe staff are highly accustomed to working with the myriad professionals across the multiple 
systems that work to protect children and support vulnerable families. Kempe staff have 
conducted focus groups, interviews, and surveys with a range of public and private child welfare 
staff (administrators, program directors, supervisors, caseworkers, information technology staff 
and analysts, administrative staff, and volunteers) at federal, state, and county levels. Given the 
interconnection between the formal child welfare system and other human service systems, the 
Kempe team has also acquired significant experience gathering information from and involving 
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other stakeholders, ineluding parents/caregivers and representatives from mental health, 
substance abuse, prevention, juvenile justice, courts, and developmental disabilities. Kempe staff 
will draw on these experiences as they collaborate with Chapin Hall, lOA and PCCi staff to 
develop the decision support and predictive analytic tools described below. 

Chapin Hall 

Given its extensive work conducting evaluations of public child welfare systems, Chapin Hall 
staff has vast expertise in analyzing the effectiveness of large government service programs 
administered through multiple funding streams. Because Chapin hall has experience evaluating 
programs at the national, state, regional, county, and private service provider level, Chapin Hall 
researchers are extremely familiar with the impact federal, state, local, and private funding 
streams have on program administration and program outcomes. Chapin Hall staff has particular 
expertise in working with the range of child welfare programs. 

Chapin Hall’s range of experience and expertise is organized around four central areas, each of 
which relates to the scope of services sought under this RFP: the use of administrative data to 
understand performance over time; conducting process studies; performing cost analyses; and, 
providing consultation and technical assistance. 

Chapin Hall also has direct experience in working with Allegheny County DHS having served as 
the evaluator for their Family Group Conference initiative. Work in other agencies includes 
several IV-e waiver evaluations including Colorado, Ohio, and as the lead evaluator for the 
Tennessee Waiver evaluation in collaboration with Kempe. Much of Chapin Hall’s work with 
public child welfare systems is done through Chapin Hall’s Center for State Child Welfare Data. 
Among Chapin Hall’s most recent accomplishments. State Data Center staff recently completed 
a comprehensive link between foster care and Medicaid data. The link is at the individual, claim, 
and placement level. The data account for $2.6 billion worth of Medicaid claims, which are 
retrievable on a person, placement/facility-specific basis. 

PCCi 

PCCi has significant experience managing information technology (I/T) projects throughout the 
east coast, with particular emphasis in public sector, and specifically within the Commonwealth 
of Pennsylvania. Our team has substantial experience deploying IBM technologies in large scale 
roll-outs and in managing projects both large and small. 

PCCi’s project management team has managed hundreds of IT implementation projects at a 
variety of agencies in Harrisburg, Philadelphia and throughout the remainder of the state. These 
projects have ranged in size from single server installations to large scale enterprise server 
deployments and SAN mirroring implementations. 

Technology Capabilities 

PCCi is an IBM Business Partner that delivers customer-centric infrastructure solutions for IBM 
and related technologies for federal, state and local government as well as commercial and 
private sectors. Additionally, the PCCi team offers the following technical services skills: AIX, 
Linux, LPAR, TSM; System p. System x, BladeCenter, PureFlex; High Availability (HACMP, 
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Veritas, MSCS); Disk and Tape Storage (V7000, XIV, DS8000); Virtualization; Storage Area 
Networks (Brocade, McData, Cisco); Disaster Recovery and Performance Tuning. 

The combined expertise of the staff from Kempe, Chapin Hall, 10A, and PCCi puts the project 
team in an excellent position to not only meet the requirements set forth in the RFP but to do so 
in a creative, high quality, efficient manner. We are very conversant with the details of social 
service delivery systems and experienced in working with public agency staff. 

3) Collaborative projects on which our organizations worked. 

The project team for this proposed work has extensive experience collaborating with each other 
and other entities. Kempe staff have a history of seeking out such collaborations, prefer to work 
collaboratively, and the majority of our projects involve collaborations with our partners and our 
clients. 

For example, in recent years, staff from the Kempe Center and Chapin Hall have collaborated on 
projects including the Texas Foster Care Redesign project, described above, are currently 
partners on a IV-E Waiver evaluation project in Tennessee, and a project examining a Predictive 
Risk Modeling application in New Zealand. Kempe recently partnered with Walter R. McDonald 
and Associates on the execution and completion of a five year cross-site evaluation under the 
auspices of the Quality Improvement Center for Differential Response. Finally, as indicated 
above, Kempe was selected to join an evaluation collaborative lead by Colorado State University 
to conduct the community response evaluation that will examine what happens to cases screened 
out from Colorado’s child protective services system. 

4) How our collaboration will manage the process of working with DHS to design and 
implement decision support tools and predictive analytics. 

As a beginning principle our team intends to work in close collaboration with the county staff 
assigned to the project. We would also propose that we coordinate project activities with a 
project manager/liaison as determined by DHS. At this point we anticipate working with research 
analysts, IT, and program staff. Our team is very experienced in working with individuals in all 
these roles in social services agencies. Ideally, as a collaboration we plan to work with these staff 
to develop a comprehensive understanding of the county’s service delivery system and data 
warehouse resources with respect to the two service populations we intend to work with. In 
addition we plan to work with these staff to conduct project planning, obtain data sets and 
associated approvals, engage or conduct joint training where appropriate, and monitor project 
activities and key deliverables. 

Because of the complexity of the proposed project, as a second principle communications are of 
crucial importance. For past projects misunderstandings have been averted by attention to 
working with agency staff to develop and finalize the project plan. This is coupled with routine 
communications during implementation including bi-weekly phone conferences where progress 
status is described, decisions are noted, and followed-up with written communication with the 
involved parties. 
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As a third principle, our project team is intent on developing tools that fit the needs of the county 
and proposing ways that decision tools can most benefit the respective county agencies. 
Therefore we are committed to an evidence informed approach to implementation. For this 
reason we think it will be important to work with the county at all stages of implementation both 
during the first year and after to constantly evaluate and document how the elements of the 
project are achieving the project goals. In addition we propose working with the county project 
manager to develop a county advisory committee consisting of key county staff and external 
advisors. This advisory group would convene at least twice during the first phase; once to review 
the final project plan, and to review the project evaluation. We believe this will help us to make 
adjustments during the implementation for the proposed work scope, but it is also needed to 
improve the effectiveness of new implementation processes. 

Finally, based on our experience with implementation we recognize that the technology and 
analytics associated with decision support tools are only half the equation. It is also critical to 
help insure that staff are prepared to use them and have the support to do so. Attention to 
implementation hinges on working with county staff to identify of the organizational drivers and 
leverage points and to formulate strategies that account for them. 

5) Describe experience or approach to working with an existing IT vendor to 
implement/integrate solutions. 

Our approach to working with the county’s IT vendors is to emphasize the enhanced capacity 
represented by the proposed decision support tools as a value added component to the existing 
and planned infrastructure. It will also entail including the vendor during the planning phase of 
the project. As important will be to include the IT vendor(s) in the project evaluation toward the 
end of the first year’s effort. Primarily, the goal of this process with the vendor is to assess 
feasibility and develop a method of integration with the technical aspects of the project and the 
county’s IT infrastructure. 

We would hope to operate in a collaborative fashion on scaling and integrating the decision 
support applications. Clearly there are cost and software licensing concerns involved in 
evaluating and planning the project. These concerns may present potential tensions during all 
phases, but especially during phases where the project would be brought to scale. Our intent if 
such tensions arise is to defer to county agency staff as the primary point of resolution. 


Project Description 

1) Project goals and objectives. 

The overall goal and proposed objectives for the project are identified below: 

Project Goal: Improve the outcomes for social service populations through client classification 
based service delivery allocation decision support models. The concept underlying this goal and 
which is described in more detail below includes decision support for both workers and for 
general service planning and resource allocation. We propose to accomplish this goal through 
three phases; 1) design and proof of concept, 2) full deployment including information system 
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integration for one or two key soeial service populations, and 3) an evaluation of the project and 
expansion to other populations if warranted. 

The project objectives are as follows: 

1. Improve the availability, reliability, validity, and timeliness of assessment data for 
clinical and resource management, 

2. Improve the turnaround and integration of assessment data into the county information 
system infrastructure to support worker service decision-making, 

3. Use integrated linked data multiple data sources including new assessment data and 
predictive modeling to classify clients and/or families into needs based clusters of major 
social services populations, 

4. Use agent based simulation techniques to forecast service needs and support resource 
allocation decision, 

5. Use agent based simulation to anticipate and monitor service delivery system quality 
indicators and outcomes 

6. Integrate the components of the proposed project into county’s information system 
infrastructure, and its workforce. 

The conceptual approach we propose to achieve the project goal and objectives is presented 
visually in Figure 1; Predictive Integrated Outcomes and Service Delivery Modeling (PIOSDM). 
The figure depicts an integrated framework consisting of three main decision support 
components: computerized adaptive testing/assessment, intelligent outcome management and 
assessment (predictive modeling), and agent based simulation. 
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A purpose of the project is to integrate the various products from the three tools. The concept is 
that data from the CAT-MH will be combined with other administrative data to support the SPSS 
modeler, these data in turn will help to define need clusters where the assessment and 
administrative data regarding service delivery for the clusters can be used to define baseline 
parameters for the simulation. For all of the tools a part of the project planning and evaluation 
process as described above will focus on how the three tools can be integrated into the current 
DHS IT infrastructure including COGNOS. However, for this project the team will be 
implementing the three components as an integrated framework for two populations as a proof of 
concept. 

Computerized Adaptive Test-Mental Health or CAT-MH is an information technology supported 
assessment and service decision support tool we propose to use with adolescents that is based on 
obtaining self-report data directly from the service population. The service participant interacts 
directly with the input device (computer, tablet, smart phone, etc.) and is prompted to respond to 
questions. The input device can be configured to allow the participant privacy while completing 
the assessment including aural questioning (via headset) in a language of choice. 

Examples of possible use by case management staff include weekly monitoring of depressive 
severity following a prescription of an antidepressant to insure that the treatment is functioning 
as intended. It should be obvious that the implementation of the CAT-MH is almost as important 
as its scientific foundation. A major advantage of CAT in this regard is that the starting value for 
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the next session can be informed by the score on the previous session leading to further reduction 
in patient and staff burden. 

The assessment results are available to the case manager, but in the PIOSDM framework it can 
be integrated with other data into the county’s database for the population. An advantage of the 
CAT-MH is that it supports both case management and Intelligent Outcome Management and 
Assessment (lOMA) processing and because it is a self-assessment potentially the agency 
workload burden associated with obtaining assessment data completed by staff. 

Intelligent Outcome Management and Assessment (lOMA) has at its core the IBM SPSS 
Modeler suite of predictive analytic tools. The SPSS Modeler is capable of obtaining integrated 
data inputs from extant information systems and data bases including Oracle. 

The SPSS Modeler software will be used to develop subpopulations of clusters defined by key 
characteristics that are statistically distinct; client needs clusters. These clusters are organized on 
the basis of characteristics drawn from the available integrated data. Assuming the success of the 
analytic process, these clusters will be used as the basis for the agent based simulation 
parameters. 

The Agent Based Simulation (Ospedale) is a forecasting tool developed through a collaboration 
of the Chapin Hall Center for Children, Argon Laboratories, with input from Kempe faculty. The 
purpose of the model is to simulate a service delivery system. When properly parameterized the 
model produces forecasts of service utilization and outcomes. The underlying simulation 
architecture is based on statistical representations of individual agents (workers, supervisors, 
mangers, etc.) who reach decisions about individual service participants. These decisions form a 
probabilistic state space and drive the dynamics of the service delivery system. Ideally the agent 
level decision making probabilities are based on parameters obtained from an analysis of the data 
derived initially from the actual service delivery system, in the case of the POISDM project these 
parameters will be derived from the identification of client clusters and an analysis of an 
historical baseline of service delivery utilization data specific for each needs cluster. Outputs of 
the process include service delivery system entries, service volumes, and service transitions (e.g., 
transitions to less or more restrictive settings), and system exits. The model is designed to 
account for different assumptions about the underlying economics of the service delivery system 
including both demand and supply based theory. 

Once simulation is parameterized and tuned to correspond to actual baseline performance, 
simulation outputs will be presented to suggest how the resources can be reallocated. This is an 
interactive process conducted by modelers and program management and involves developing 
use scenarios that are then tested to gauge forecasted performance, identify missing data or 
analyses, and support decisions about the reallocation of resources. Finally, the model is used to 
evaluate the performance of the system against the forecasted baselines. 
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2) Detailed Description of the Services and Consultation to be Provided 

Overview 

The service proposed here is to develop a design and proof of concept during the first year of 
funding. Upon achieving a successful proof of concept our collaborative is prepared to work with 
DHS to fully integrate and scale the modeling approach into the county’s information system 
infrastructure. Our proposed plans for the first year are described below in detail and plans for 
possible renewal are described more broadly. 

Conceptual Approach 

As described above the conceptual approach we propose is based on adapting and evaluating 
three decision support technologies: 1) computerized adaptive testing/assessment, 2) intelligent 
outcome management and assessment (predictive modeling), 3) and agent based simulation. 

Each of these technologies constitute relatively advanced IT and analytic tools which in and of 
themselves would have value in the decision support context of the county. Taken together our 
team believes that these technologies will constitute a major advance in decision support for 
social services, and could serve as a model for other populations and other jurisdictions. Our 
team is prepared to offer services that will implement the technology, while at the same time 
working collaboratively with the DHS staff in all aspects of the project. 

Target Populations 

For this project we are planning on a design and implementation for two key populations; 
adolescents in the foster care system, and elderly with mental health risks. There are several 
reasons for focusing on these populations for the design and proof of concept. First, our project 
team is very familiar with these populations and their service delivery milieus. Given the short 
time frame for the initial project activity, this will allow our team to be at its most productive and 
take advantage of this knowledge rapidly to assess the existing data resources, develop predictive 
analytics based on knowledge about the populations and their underlying risks, and being in a 
position evaluate and adjust the implementation based on our experiences with similar projects. 

Second, aspects of the existing versions of the technology (particularly the CAT-MH and agent 
based simulation) have already been developed with these populations in mind. This means that 
our team know what staff in addition to the IT and research personnel at DHS need to be 
engaged in the implementation and how to carry out work with those audiences in mind. Finally, 
the populations are of a modest and manageable size, but still large enough to be appropriate 
groups for the use of probability based tools. 

Adolescents in Foster Care 

This population presents considerable challenges for most child welfare agencies. Typically 
compared to younger children in care, adolescents are more difficult to reunify with birth 
parents, more difficult to adopt, and may not be well prepared to leave foster care as they age 
out. Often these are children who experience the longest stays in care, and as the two conditions 
are related, larger number of placements while in care. Such children are often diagnosed with 
mental health needs and their use of related medications is often greater as well. Finally, they are 
exposed to and may participate in various high-risk behaviors such as abuse of substances. 


Decision Support Tools and Predictive Analytics in Human Services Kempe Center Bid 


14 



Elderly Who are Psychiatrically Hospitalized or at High Risk of Hospitalization 
This population we intend to foeus on among the elderly represents the high risk group with 
mental health needs who are psyehiatrieally hospitalized or at risk of hospitalization. Given 
these eonditions the development of low workload impaet tools as described to assist in 
diagnosing elderly with mental health needs and monitoring their progress have definite potential 
for improving effective service delivery. 

Being able to more clearly differentiate members of key high cost high risk populations with 
varying needs through predicative modeling has important implications for defining the 
appropriate range of evidence based clinical services, estimating the potential volume of needs, 
and allocating resources more optimally based on these needs. 

Consultation Activities 

The activities of the project consist of a planning and evaluation activity and the consultation for 
the three decision support tools. These activities are a design and proof of concept and are 
intended to take place over no more than one year. 

Phase One Plan and Evaluation 

During the planning and evaluation activity the proposed work scope will address the 
development of more detailed identification of the project activities and schedule. Included in 
these areas are startup activities, identification of participating county staff and other 
stakeholders, communications and logistics, advisory groups, project ethics, software licensing 
and installation, training, testing participants and schedules, and other tasks related to the 
management and support of the project. Kempe Center staff will largely be responsible for this 
aspect of the project. 

Task areas for the project planning activity include assessing the current data resources available 
to the department. This includes understanding the cross agency availability of linked data and 
the specifications of the individual databases that are associated with the focus populations. 

It will also be necessary to develop an understanding of the current IT infrastructure, particularly 
regarding capacity and resources of the county data warehouse, hardware, networking, server 
capacity, and what software is available. We are aware that tools such as COGNOS and SPSS 
are in use and of course these are directly with the tools we are proposing. Equally important is 
to identify appropriate staff within DHS who are able to work with our team to process data, 
provide linkages to data resources, and to address any concerns about data ethics and security 
concerns. In other projects, our team has worked with a variety of encrypted data exchange 
protocols including direct access via VPN. We are prepared to work through the options and 
develop methods for data exchange that meet the requirements of the county. Once the final 
project plan is drafted we would propose presenting it to a project advisory group for further 
input and feedback before it is finalized. 

This activity area also includes evaluation of the project with the county staff in the final months 
of the first year to assess whether the proof of concept is sufficiently robust and if it is feasible to 
implement on a full scale basis. This is also a point to determine whether other populations 
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served by the DHS might benefit as well. As our team envisions it the project evaluation would 
hinge on the following questions: 

1. Was it possible to introduce the three project components into DHS? 

2. Were the appropriate program/clinical staff able to use the CAT-MH? 

3. Was the project team able to develop robust needs clusters for the focus populations 
using lOMA? 

4. Was the team able to develop and test appropriate resource allocation models for the 
focus populations using agent based modeling? 

5. What IT infrastructure changes would facilitate scaled implementation? 

6. Is scaled implementation feasible, and if so for which populations? 

Computerized Adaptive Testing/Assessment 

The consultation for this aspect of the project would be carried out primarily under auspices of 
Chapin Hall and supported by our consultant Robert Gibbons who is the author of the CAT-MH, 
and with some coordination by Kempe staff As described above the CAT-MH is designed to be 
used directly by the client, in this case adolescents in or newly entering the foster care system, or 
elderly clients at risk of psychiatric hospitalization. Because the CAT-MH is immediately 
available for use with internet capable computing much of the activity will focus on 
implementing the assessment process in the DHS context. The consultation for this activity area 
will include the following: 

1. Determining physical settings can be used for proof of concept testing where participants 
can interact with the CAT-MH and setting up the CAT-MH in. 

2. Arrange for any additional hardware and hardware configurations needed to set up the 
CAT-MH through PCCi (e.g., headphones with microphones, tablet computers, etc.) in 
the assessment settings^. 

3. Training for involved county staff to work with participants. 

4. Developing and implementing the assessment protocols including consents and 
approvals, assessment scheduling, post assessment, and follow-up. 

5. Training involved staff to interpret the CAT-MH results 

6. Integrating the assessment results with the DHS participant databases. 

7. Monitoring the implementation to assess levels of participation, error reporting, and other 
data quality concerns. 

8. Review of predictive analytic results (see below). 

Finally, the project team will make recommendations regarding bringing the computerized 
adaptive assessment to scale as part of the overall recommendations for the project going 
forward. 

Predictive Modeling for Classification 

The predictive modeling work of lOA team will be responsible for implementing a repeatable 
predictive modeling process, and will be the lead predictive model developer. In the first phase 
of work lOA will lead the predictive analytic model development and testing for the foster care 
and elderly populations. In collaboration with Kempe, lOA will clean, structure, pre-process, and 

^ Note: a limited budget is available for these items. 
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model the available data. In addition, lOA will engage DHS staff and administrators in a 
qualitative proeess using foeus groups to vet and analyze the models and rule sets that have been 
developed. 

The lOA team will additionally eollaborate in all phases of the projeet to ensure that the 
predietive analytie models are connected to DHS dashboards and assessment tools as seamlessly 
and elegantly as possible. PCCi will provide and oversee installation of the IBM SPSS Modeler 
as the authorized IBM vendor for the software. 

Agent Based Simulation and Resource Allocation 

As described above the agent based simulation model component of the project is oriented to 
improving program resource definition and allocation. This part of the project is based on the 
Ospedale modeling environment originally designed to understand service delivery for children 
in group or residential care (see description below and attachment) and will largely be carried out 
by staff from Chapin Hall with assistance from Kempe Center staff We propose adapting the 
model to address the foster care population including those in residential care. Services for the 
agent based model are divided between three main activities; 1) obtaining data to develop basic 
service delivery models, 2) tuning the model against baselines, 3) incorporating sub-populations 
of needs based clusters derived from the predictive modeling, and 4) development of the 
resource allocation dashboard. 

3) How the program will be integrated with other information technology and tools. 

The tools proposed for use in the project can all be integrated into the DHS IT infrastructure. For 
example, the CAT-MH is internet based using conventional browser technology. Products from 
the CAT-MH process can uploaded into the appropriate client records with appropriate support 
from county IT analyst staff. As described above the IBM SPSS modeler is designed to be 
compatible with standard data base architecture. SPSS Modeler uses a client/server architecture, 
allowing resource-intensive operations to be executed on a server, and will readily support 
application in COGNOS which already in use by DHS and maintained by Deloitte and Touche. 
The current version of Ospedale was designed to work with Windows. 

4) Timeline for design and development 

The general activities were described above are shown in Figure 2 as a Gantt chart showing the 
overall time frames for activities during the first year of the PIOSDM project. 
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Figure 1 Phase One Tasks and Timeline 


ID 

Task Name 

Start 

Finish 

0 

PfCfctM Integrated Outcomes and Service Delrvery Modeling 

9/1/14 

10/14/15 

1 

1 Phase One - Design and Testing (Year One) 

9/1/14 

10/14/15 

2 

1.1 Final Phase One Plan 

9/1/14 

10/14/15 

3 

1.1.1 Assess Data Base Resources 

9/1/14 

10/20/14 

4 

1.1.1.1 Foster Child Care 

9/1/14 

9/30/14 

5 

1.1.1.2 Elderly Care 

10/1/14 

10/20/14 

6 

1.1.2 Finalize Time Frame 

9/15/14 

10/14/15 

7 

1.1.3 Establish and Convene Review/Advisory Committees 

11/1/14 

8/31/15 

6 

1.1.4 Implement IRB and Address HIPAA 

9/15/14 

11/15/14 

9 

1.1.5 Evaluate Phase One 

6/V15 

8/31/15 

10 

1.2 Foster Care Adaptive Assessment Pilot 

10/1/14 

8/31/15 

11 

1.2.1 Initial Assessment Set Up with Foster Care Staff 

10/1/14 

11/30/14 

12 

1.2.2 Allegheny Staff Training 

11/15/14 

7/15/15 

13 

1.2.3 Institute and Maintain Pilot Assessment System 

U/1/14 

7/31/15 

14 

1.2.4 Process Evaluation of Assessment System 

1/15/15 

8/31/15 

IS 

1.3 Predictive Modeling and Qassification 

9/15/14 

8/15/15 

16 

1.3.1 Source Data Specification and Retrieval 

9/15/14 

12/31/14 

17 

1.3.2 Integrate Adaptive Assessment Data for RC 

5/1/15 

7/31/15 

18 

1.3.3 Generate initial Oient Clusters 

V2/15 

4/30/15 

19 

1.3.4 Generate Oient Ouster w Adaptive Assessment Data 

6/1/15 

8/15/15 

20 

1.3.5 Training for Allegheny Analysts 

V2/15 

8/15/15 

21 

1.4 Agent Based Simulation 

9/15/14 

8/lS/lS 

22 

1.4.1 Source Data Specification and Retrieval 

9/15/14 

12/31/14 

23 

1.4.2 Estimate Initial Parameters for Foster Care 

1/2/15 

3/15/15 

24 

1.4.3 Generate Initial Simulation Models for RC 

3/16/15 

4/30/15 

2S 

1.4.4 Estimate Parameters for RC Ousters 

3/20/15 

5/31/15 

26 

1.4.5 Generate Model for RC Clusters 

6/1/15 

8/14/15 

27 

1.4.6 Develop and Test Resource Allocation Scenarios 

7/1/15 

8/15/15 


Q1*15 Q2MS Q3‘i5 Q4*15 

Uujswloak^l 


If the proof of concept is successful and some or all aspects deemed appropriate for scaling, our 
team proposes conducting the following broadly oriented tasks in years two and three; 


Overview Tasks for Years 2 and 3 

Phase Two - Deployment and Integration (Year Two - Three) 

Integrate Projeet Components Into IT Infrastrueture 

Provide ongoing Training 

Refine/Refresh Predietive Models 

Foster Care 

Elders with Mental Health Needs 

Refine Simulation Models 

Foster Care 

Elders with Mental Health Needs 


Phase Three - Modeling Evaluation and Next Steps (Year Two & Three) 

Assess Applicability to Other Populations 

Pilot Implementation 

Monitoring and Effectiveness 

Support Ongoing Integration for Other Populations 


Decision Support Tools and Predictive Analytics in Human Services Kempe Center Bid 


18 








































































5) Examples of tools/systems already developed. 

Computerized Adaptive Test/assessment (CAT-MH) 

The CAT-MH, is currently a collection of three adaptive tests for depression, anxiety, and mania, 
and a diagnostic screening test for major depressive disorder (CAD-MDD) developed as part of a 
5-year grant from the National Institute of Mental Health. The CAD-MDD produces a screening 
diagnosis of depression and a corresponding confidence level associated with that diagnosis. A 
more detailed description of its development, operation, and evaluation are found in the 
attachments to this proposal. 

The ultimate goal of the CAT-MH is to make diagnostic screening of mental disorders such as 
depression and long term monitoring of the severity of those disorders as seamless as possible, 
decreasing patient and clinician burden to near zero levels. The CAT-MH will be distributed as 
a cloud-computing system where a secure and HIPAA compliant internet system is used to 
assess individuals anytime from anywhere on any internet-capable device. For example, the 
cloud will permit an individual in urban Chicago, the most remote corner of the Navajo Nation 
or rural West Virginia to log on 24x7x365 via an internet browser with a mobile computing 
device, a laptop or a desktop to a secure website with a unique ID and password and carry out the 
assessment. The results of that assessment and automated suicide warnings can be directly 
transmitted to any number of reviewers in near real time and can be placed directly into the 
participants data file. 

IBM SPSS Modeler 

IBM SPSS® Modeler Professional is a data mining workbench used to analyze structured data to 
create predictive intelligence. SPSS Modeler Professional is popular with analysts and business 
users alike. Its automated data preparation and modeling features enable non-analysts to produce 
accurate models quickly and easily without specialized skills. Professional analysts can also take 
advantage of the software’s advanced data preparation and predictive modeling capabilities to 
create the most sophisticated of streams. The intuitive graphical interface of SPSS Modeler 
makes it easy for users to visualize every step of the data mining process as part of a “stream.” 

By interacting with these streams, analysts and business users can collaborate - adding business 
knowledge and domain expertise to the data mining process. It allows them to focus on 
discovering insights rather than on technical tasks like writing code. They can also pursue “train- 
of-thought” analysis, explore the data more deeply; uncovering additional relationships that 
make sense to the organization. 

From this visual interface, one can easily access and integrate data in virtually any type of 
database, spreadsheet or flat file, including IBM SPSS Statistics, IBM SPSS Data Collection, 
Cognos Business Intelligence, SAS and Microsoft Excel files. When combined with SPSS 
Modeler Professional Server, there is no need to move data from large databases, since the 
analytics and mining take place in-database. The result is a significant improvement in analytical 
performance. 

Agent Based Simulation (Ospedale) 

Ospedale, an agent based simulation model, was originally formulated to address service 
delivery questions and simulate the aggregate experiences of children in group or residential 
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care. Chapin Hall developed Ospedale to utilize data derived from administrative data sets to 
parameterize the agent based simulation environment specifie to the target agency. The model 
can be adjusted by making different assumptions about these parameters, for example foster eare 
entry and exit rates. In addition the model ean be tuned to operate with subpopulations and those 
can be simulated as subpopulations or in the aggregate with other subpopulations. There are 
three distinet categories of dynamic simulation models that are now built and operate in the 
Ospedale modeling environment. 

1. Models of deeisions to admit ehildren in need of eare to one of the eare path alternatives, 
based upon historical data and corresponding statistical models of admissions. 

2. Models of changes in licensed system capacity (i.e., number of beds) of facilities due to 
endogenous and exogenous factors, again based on historical data and statistical models. 

3. Models of clinical outcomes, including duration of stay, as a function of individual 
patient characteristies, demographies, family background, medical history, etc., and the 
profile of care that the child was given. 

The attaehed exhibits for eaeh of the components provide more detailed summaries of the three 
already developed tools. 

6) Staffing plan: Staff roles and qualifications. 

Overview of Project Staffing 

The proposed project team ineludes professionals with eonsiderable experience in developing 
and implementing decision support tools and in conducting research in the area of social services 
delivery and management. Many members of the team have worked for deeades assisting soeial 
services ageneies in developing their programs, implementing deeision support tools, program 
evaluation, data collection, administrative data construction, and data analysis. 

The team consists of staff and faculty from two university based organizations who are the 
primary collaborators on the proposed projeet: The Kempe Center for the Prevention of Child 
Abuse at the University of Colorado School of Medicine, and Chapin Hall at the University of 
Chicago and lOA will be form the management team for the project. Kempe will act as the single 
point of contaet and coordinate serviees. In addition the projeet will include some key 
consultants who bring considerable expertise in CAT-MH and knowledge of elder services. The 
project organizational chart is show in Figure 3. 
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Figure 2 Project Organizational Chart 



Project Roles 

Kempe Center 

John Fluke, Ph,D, is Associate Director of System Research and Evaluation at Kempe. For this 
project Dr. Fluke will carry out the role of collaboration team lead and project CO-PI for Kempe. 
Dr. Fluke’s work focuses on social service delivery system research in child welfare and mental 
health services for children. Dr. Fluke is internationally recognized as a researcher specializing 
in assessing and analyzing decision-making in human services delivery systems. Dr. Fluke also 
works on national child maltreatment data collection systems and analysis. Dr. Fluke is known 
for his innovative and informative research in the areas of child maltreatment prevalence, child 
welfare administrative data analysis, workload and costing, and performance and outcome 
measurement for children and family services. 

Dana Hollinshead, Ph,D, is an assistant professor with Kempe and has over twenty years of 
experience conducting research, policy analysis, and program evaluation in the field of child 
welfare. Dr. Hollin s head will take the role of lead project analyst and support predictive 
modeling for the child welfare populations. Dr. Hollinshead’s areas of expertise include re¬ 
reporting and recurrence of child maltreatment, differential response, performance measurement, 
system reform, and the influence of policies, practices, and demographics on child welfare 
decision-making and agency outcome measures. 
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Matthew Nalty is an information specialist at Kempe and will serve as the project Database 
Analyst. In that role he will consult with senior leadership, subject matter experts, and partnering 
organizations to develop information lifecycle management strategies, data sharing strategies, 
and analytical or reporting needs. Coordinate planning for the automation needed for standard 
reporting that includes test plans and support processes. Document baseline business processes, 
systems, information, and technologies. Perform research to formulate data governance 
recommendations (data ownership; management; usage rights; and decision making processes, 
procedures, and mechanisms). Contribute to the development of data management strategies, 
policies and best practices through participation in management meetings and other direct 
contributions. 

Chapin Hall 

Fred H, Wulczyn, Ph,D, is a Senior Research Fellow at Chapin Hall. For this project he will be 
the project CO-PI and lend his expertise and experience in agent based modeling. He is the 2005 
recipient of the National Association of Public Child Welfare Administrators’ (NAPCWA) Peter 
Forsythe Award for leadership in public child welfare. In 2011, he was honored with the Flynn 
Prize, an award given to a researcher whose work has had a demonstrable impact on vulnerable 
populations. Among his responsibilities at Chapin Hall, Dr. Wulczyn is director of the Center for 
State Child Welfare Data, a collaboration of Chapin Hall, the American Public Human Services 
Association, and other research partners. Dr. Wulczyn has designed two major social 
experiments: the Child Assistance Program and the HomeRebuilders project. The Child 
Assistance Program was awarded the Innovations in Government Award from Harvard 
University and the Ford Foundation. Dr. Wulczyn earned his Ph.D. at the University of Chicago. 

Robert Gibbons, Ph.D, is the developer of CAT-MH and will serve as our consultant for this 
aspect of the project. Dr. Gibbons is a Professor in the Departments of Medicine & Health 
Studies at the University of Chicago. Among his many research interest is item response theory 
and computerized adaptive testing, the basis for CAT-MH. He will work closely with Chapin 
Hall to implement the CAT-MH and provide ongoing input into the use of CAT-MH in the 
development of the needs based clusters. 

lOA 

David Schwartz - David R. Schwartz, MSW is Chief Executive Officer at Intelligent Outcome 
Assessment (10A), and is the predictive analytic modeling team lead and project CO-PI for lOA. 
Schwartz's work focuses on rigorously building, testing, and deploying predictive analytic 
models in database or through analytic servers using social service data warehouses. Schwartz is 
one of the first researchers to train and test predictive analytic modeling algorithms in 
collaboration with interdisciplinary researchers using child welfare, juvenile recidivism, and 
foundation program evaluation data (e.g., Schwartz, Kaufman, Schwartz, 2004; Schwartz, Jones, 
Schwartz, & Obradovic, 2008), and served as data modeling lead for several cutting edge 
evaluation and predictive analytics research projects to better understand the pathways to 
successful program outcomes for youth, families, volunteers, and the elderly (e.g.. Gap 
Foundation, Thrive Foundation for Youth, Tufts University Health Plan Foundation, Target, 
Feeding America, Encore Senior Centers, Girl Scouts, and New York State Office of Children 
and Eamily Services). Schwartz was a case manager for the Choice Program, an alternative to 
secure detention program with a significant number of crossover youth. 
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CO - PI and Lead Predictive Modeler 

Jonathan Prince, Ph,D, - Prince will collaborate with the team on mapping DHS decision 
points, assessing the available integrated data for modeling, and providing frequent consultation 
during all phases of model development and deployment. 

Dr. Prince will also help scope and articulating the practice implications for decision support 
tools for the elderly (primary role) and youth in foster care (secondary role). 

PCCi 

Debra Pierson, PMP, will serve as Project Executive for this project, providing project oversight 
and manage contractual matters. Deb Pierson has more than twenty five years experience 
providing deployment services to the Commonwealth. Deb performed as project manager for the 
installation of more than 10,000 PCs for the County Assistance Offices for the PA Department of 
Welfare, and provided oversight for the installation of 31,000 laptops for the Classrooms for the 
Future project. In addition. Deb supported the PCCi project management team for the Detroit 
Public Schools 40,000 netbook installation project. 

Janese Shenk will serve as the primary PCCi contact for the project and support the IBM SPSS 
Modeler software. Janese has more than ten years of experience providing project management 
and technical services to the Commonwealth of Pennsylvania, including management of similar 
rollout projects. Janese managed the installation of over 20,000 laptops and 2000 interactive 
whiteboards in more than 60 school districts in Central Pennsylvania over fourteen weeks as part 
of Year 2 of the Department of Education’s Classrooms for the Future project. PCCi had up to 
five installation teams performing simultaneously under Janese’s direction during this project. 

Project Advisors 

Nick Intintolo, IBM’s National Industry Specialist for Social Services/Criminal Justice will 
serve in a non-billing oversight role to advise the project team, and when available, participate in 
customer meetings. Intintolo spearheaded the IBM Intelligent Outcome Management and 
Assessment (lOMA) Framework, while orchestrating a collaboration of thought leaders and 
scholars from across public sector industry sectors, to bridge the gap between best-in-class 
technology manufacturing, and world class domain expertise. He has spent over twenty years 
advising U.S. Government leaders adopt state-of-the-art technology solutions in the social 
services, criminal justice, and public safety domains, most notably an original contributor to the 
PA-JNET architecture. 

Emily Putnam-Hornstein joined the faculty of the University of Southern California in 2011 
after completing her doctoral studies at the University of California, Berkeley. With an interest in 
child maltreatment, public child welfare systems, and extensive experience in administrative data 
analysis, Putnam-Hornstein’s current research focuses on the application of epidemiological 
methods to improve the surveillance of non-fatal and fatal child abuse and neglect. Analysis of 
this repository has generated knowledge as to where scarce child welfare resources may be most 
effectively targeted and advances an understanding of maltreated children within a broader, 
population-level context. 
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3. BUDGET AND BUDGET DESCRIPTION 

The budget for the project consists of personnel costs, travel, subcontractors, software, and 
indirect costs as follows: 

Personnel Costs - Personnel costs for Kempe are based on an hourly rate that reflects salary and 
benefits. 

Travel Costs - Travel reflects estimated coach round trip coach airfare, hotel, and per diem 
estimated on the basis of trips with 3 or 4 day overnights. This includes travel for project 
personnel to Pittsburg, Pennsylvania from Denver, Colorado; Chicago, Illinois; and Philadelphia, 
Pennsylvania. 

Subcontracts - The subcontract line items reflect the total personnel related costs (salary and 
benefits), consultants (Dr. Gibbons at Chapin Hall, and Dr. Prince with 10A), travel, and indirect 
costs for Chapin Hall and Intelligent Outcomes Analytics (10A). This includes travel for project 
personnel to Pittsburg, Pennsylvania from Chicago, Illinois ($4,000 for Chapin Hall); and 
Philadelphia, Pennsylvania ($8,400 for lOA). 

Other Costs - Under other costs is the quote for IBM SPSS Modeler software to be supplied by 
our MWBE firm PCCi who is also a qualified IBM vendor. The quote reflects the cost for three 
production licenses, two to be used by the project team, and one to be used by Allegheny DHS. It 
also includes the cost of a 140 user IBM SPSS Modeler non-production license for the DHS. The 
licenses will be invoiced directly to Allegheny DHS by PCCi. The cost of these licenses can 
potentially be reduced if the county can work with IBM to participate in its government 
discounting program. 

Indirect - The University’s indirect cost rate on contracts through auxiliary accounts is 10% and 
encompasses Departmental/organization administration costs each contract should bear. 

The summary budget appears below; 
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BUGET SUMMARY 


PREDICTIVE INTEGRATED OUTCOMES AND SERVICE DELIVERY MODELING 


Total Personnel 


OTHER DIRECT COSTS 

Travel 

Subcontracts 

Chapin Hall 

Intelligent Outcome 
Analytics 

PCCi -IBM SPSS 
Modeler 

Direct invoice to 
Allegheny 
DHS 


Other Direct Costs 

Total Direct Costs 

Indirect cost rate at 10% 

TOTAL PROJECT 
COSTS 


PERSONNEL 

ROLE 

Hours 

Hourlv 

Rate 

John Eluke 

CO/PI 

326 

■ 

Dana Hollinshead 

Senior Research Analyst 

530 

■ 

Matthew Nalty 

Database Analyst 

452 

■ 

TBD 

Administrative Support 

136 

■ 

TBD 

Senior Research Analyst 

320 

■ 


Cost 



1764 
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4. ATTACHMENTS 


Required Forms 

Appendix B - Abbreviated Application 
Appendix C - Internal Revenue Service W-9 
Appendix D - Supplier/Remit to Information 
M/W/DBE Participation Form 
Pierson Computing WBE Reverification 

Resumes 

The Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 
John D. Fluke 
Dana M. Hollinshead 
Matthew J. Nalty 

Chapin Hall at the University of Chicago 
Fred H. Wulczyn 
Robert Gibbons 

Intelligent Outcome Analytics, LLC 
David R. Schwartz 
Jonathan D. Prince 

Tool Descriptions 

Overview of Computerized Adaptive Mental Health Testing 
lOMA - Intelligent Outcome Management and Assessment 
IBM SPSS Modeler Professional 

IBM SPSS Modeler Server performance and optimization 

Agent-Based Social Simulation in the Service of Child Welfare Issues: Defining the Demand for 
Group and Residential Care 

Letters of Commitment 

Chapin Hall 

Intelligent Outcome Analytics, EEC 
Pierson Computing Connection, Inc. 
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Required Forms 

Appendix B - Abbreviated Applieation 
Appendix C - Internal Revenue Service W-9 
Appendix D - Supplier/Remit to Information 
M/W/DBE Participation Form 
Pierson Computing WBE Reverification 
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APPENDIX B 

ABBREVIATED APPLICATION 
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income tax (if individual)? 
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APPENDIX C INTERNAL REVENUE SERVICE W-9 
A fill-in version of this form can be obtained at the IRS website. 


.. W-9 

(Rev, October 2007) 
OeiMilment of Ihe Treasury 
InlemaJ Revenue Service 


Request for Taxpayer 
Identification Number and Certification 


Give form to the 
requester. Do not 
send to the IRS. 


Name (as shov/n on your income tax return) 

Regents of the University of Colorado 



03 

03 

CO 


Business name, if different from above 

The Kempe Center 

Check appropriate box: O tndividual/Sole proprietor D Corporation Q Partnership 

C Limited liability company. Enter the tax classification (Dsdisregard^ ©OJity. C=corporatipn, P=partpership) ► .. 

Q other► State Institution oT Higher HSucation 

gj. Exempt 
payee 

Address (number, street, and apt. or suite no.) 

13123 E. 16th Ave B390 

Requester's name and address (optional) 

City, state, and ZIP code 

Aurora, CO 80045 




List account number($) here (optionaO 


Parti 


Taxpayer Identification Number (TIN) 


Enter your TIN in the appropriate box. The TIN provided must match the name given on Line 1 to avoid 
backup withholding. For individuals, this is your social security number jSSN). However, for a resident 
alien, sole proprietor, or disregarded entity, see the Part I instructions on page 3. For other entities, it is 
your employer identification number (EIN). If you do not have a number, see How to get a TIN on page 3. 
Note. If the account Is in more than one name, see the chart on page 4 for guidelines on whose 
number to enter. 


Part II 


Certification 


Social security number 


or 

Employer identification number 



Under penalties of perjury. I certify that: 


1. The number shown on this form is my correct taxpayer identification number (or t am waiting for a number to be issued to me), and 

2. I am not subject to backup withholding because: (a) I am exempt from backup withholding, or (b) I have not been notified by the Internal 
Revenue Service (IRS) that I am subject to backup withholding as a result of a failure to report all interest or dividends, or (c) the IRS has 
notified me that I am no longer subject to backup withholding, and 

3. I am a U.S. citizen or other U.S. person (defined below).. 


Certification instructions. You must cross out item 2 above if you have been notified by the IRS that you are currently subject to backup 
withholding because you have failed to report all interest and dividends on your tax return. For real estate transactions, item 2 does not apply. 
For mortgage interest paid, acquisition or abandonment of secured property, cancellation of debt, contributions to an individual retirement 
arrangement (IRA), and generally, payments other than interest and dividends, you are not required to sign the Certification, but you must 
provide your correct TIN. See the instructions on page 4. 




Sign 

Here 


Signature of 
U.S. person ^ 


VA?A 


General InstrUCtionShaun P McMullin, Deputy ConlrdBel'nition of a U.S. person. For federal tax purposes, you are 
_ , *. , s , -j . considered a U.S. person if you are: 

Section references are to the Internal Revenue Code unless 
otherwise noted. 


Purpose of Form 

A person who is required to file an information return with the 
IRS must obtain your correct taxpayer identification number CUN) 
to report, for example, income paid to you, real estate 
transactions, mortgage interest you paid, acquisition or 
abandonment of secured property, cancellation of debt, or 
contributions you made to an IRA. 

Use Form W-9 only if you are a U.S, person (including a 
resident alien), to provide your correct TIN to the person 
requesting it (the requester) and. when applicable, to: 

1. Certify that the TIN you are giving is correct (or you are 
waiting for a number to be issued). 

2. Certify that you are not subject to backup withholding, or 

3. Claim exemption from backup withholding if you are a U.S. 
exempt payee. If applicable, you are also certifying that as a 
U.S. person, your allocable share of any partnership income from 
a U.S. trade or business is not subject to the withholding tax on 
foreign partners’ share of effectively connected income. 

Note. If a requester gives you a form other than Form W-9 to 
request your TIN. you must use the requester’s form if It is 
substantially similar to this Form W-9. 


• An individual who is a U.S. citizen or U.S. resident alien, 

• A partnership, corporation, company, or association created or 
organized in the United States or under the laws of the United 
Stales. 

• An estate (other than a foreign estate), or 

• A domestic trust (as defined in Regulations section 
301.7701-7). 

Special rules for partnerships. Partnerships that conduct a 
trade or business in the United States are generally required to 
pay a withholding tax on any foreign partners' share of income 
from such business. Further, in certain cases where a Form W-9 
has not been received, a partnership is required to presume that 
a partner is a foreign person, and pay the withholding tax. 
Therefore, if you are a U.S. person that is a partner in a 
partnership conducting a trade or business In the United States, 
provide Form W-9 to the partnership to establish your U.S. 
status and avoid withholding on your share of partnership 
Income. 

The person who gives Form W-9 to the partnership for 
purposes of establishing its U.S. status and avoiding withholding 
on its allocable share of net income from the partnership 
conducting a trade or business in the United States is in the 
following cases: 

• The U.S. owner of a disregarded entity and not the entity. 


Cat. No. 10231X 


Form W“9 (Rev. 10-2007) 
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APPENDIX D 

Industry Classification by NAICS Code 

Primary industry 

Secondary Industry (if applicable) 


*lf code ]s not known go to http://www.census.eov/eoccl/naic502/naicod02.htm and select the correct code. 



Required Information 

Supplier/Remit To Information (Search Type "V"l — (where check will be mailed for payment. Check must be 
made payable to exact name listed under TIN provided or check cannot be processed.) 

Please print or type 


Supplier/Payee Name 


Regents of the University of Colorado 


Address Line 1 The Kempe Center 

Address Line 2 13123 e leth Ave B 390 


Address Line 3 


City 


Aurora 


State 


CO 


ZIP Code 


80045 


Telephone Number 303 - 864-5338 


Fax Number 


303-864-5367 _ 

*lf the "remit to" information provided on form does not match invoices submitted for payment, the Controller's 
Office MUST contact supplier to verify address information before payments are processed. Thank you for your 
cooperation. 
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APPENDIX D 


If the Allegheny County Department with which you do business is known, providing the information 
below will help in the processing of your payments. Failure to include the information may result in 
processing delays. 

Allegheny County 


Departmental Contact 

Supplier/Payee Contact Name 

Name 

Leslie Lewis-Pollard 

Name 

Lorena Hendrix 

Telephone 

No. 

412-350-5663 

Telephone No. 

303-864-5338 

Fax No. 

412-350-3414 

Fax No. 

303-864-5367 

EMail 

Address: 

Llewis- 

pollard@alleghenycounty.us 

Email Address: 

lorena.hendrixOchildrenscolorado. 
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_ COUNTY OF ALLEGHENY _ 

M/W/DBE PARTICIPATION STATEMENT 
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Pennsylvania 

DEPARTMENT OF GENERAL SERVICES 


July 11,2013 


Debra Pierson 
President 

Pierson Computing Connection 
10 Long Lane, Suite 100 
Meehan icsburg, PA 17050 


Certification Number: 154433-2012-07-8 

Expiratioji Date: July 23,2014 

Business Tj'pes: Information Technologj' 


Dear Debra Pierson; 

The Bureau of Small Business Oppoilunities has reviewed your request for verification along with 
your suppoiting documents. 1 am pleased to inform you that your company has been verified as a 
Woman Business Enterprise (WBE) based on your certifications with the Pennsylvania Unified 
Certification Program (UCP) and the Women’s Business Enterprise National Council (WBENC) 
and your small business self-certification registration through the Small Business Procurement 
Initiative (SBPl). The Commonwealth now recognizes your company as a Small Diverse Business 
(WBE). 

Please understand that the information that you provided for verification was represented as being 
true and correct. Please know that tiie Commonwealth shall treat any misstatement as fraudulent 
concealment of the true facts punishable as a crime under Section 4904 of the Pennsylvania Crime 
Code, Title 18, of Pa. Consolidation Statutes relating to unsworn falsification to authorities 

Your verification is valid utitil the expiration date noted above. For additional information or 
assistance, please contact Bureau staff at (717) 783-3119 or by email at ra-bsboverification@pa.gov. 
Best wishes in your business endeavors. 

Sincerely, 

DeShawn A. Lewis, Director 

Bureau of Small Business Opportunities (BSBO) 


Bureau of Small Business Opportunities 
401 North Street | Room 611 | North Office Building | J larrisburg, PA 17120 



Resumes 

The Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 
John D. Fluke 
Dana M. Hollinshead 
Matthew J. Nalty 

Chapin Hall at the University of Chicago 
Fred H. Wulezyn 
Robert Gibbons 

Intelligent Outcome Analytics, LLC 
David R. Schwartz 
Jonathan D. Prince 
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John D. Fluke 


CURRICULUM VITAE 


PERSONAL HISTORY AND BIOGRAPHICAL SKETCH: 

Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, Department of 
Pediatrics, School of Medicine University of Colorado, Anschutz Campus, Aurora, CO 
Associate Director, Systems Research and Evaluation and Research Associate Professor (2012 to 
present) 

The Gary Pavilion at Children’s Hospital Colorado I Anschutz Medical Campus 
13123 East 16th Avenue, B390 I Aurora, CO 80045 
Phone: (303) 864-52191 Fax: (303) 864-5367 
iohn.fluke@ucdenver.edu 

Dr. Fluke has over 33 years of experience in social service delivery system research in the area of Child 
Welfare and Mental Health Services for children. He is internationally recognized as a researcher 
specializing in assessing and analyzing decision making in human services delivery systems. He is also 
active in the area of national child maltreatment data collection systems and analysis and has worked 
with data collection programs in the Balkans, Canada, Saudi Arabia, the US, and for UNICEF. He has 
conducted research and evaluation at all levels of government, in the private not-for-profit sector, and 
with national foundations and associations that includes work both in the U.S. and internationally. He is 
also known for his innovative and informative research and evaluation work in the areas of child 
maltreatment prevalence, child welfare administrative data analysis, workload and costing, and 
performance and outcome measurement for children and family services. 

At the Kempe Center Dr. Fluke will pursue ongoing research in the area of child welfare decision 
making, administrative data analysis, and the implementation and scale up of evidence supported 
interventions. He will also pursue the development child maltreatment epidemiology including 
applications to international monitoring and evaluation activities. He holds a senior position as a 
research faculty member, and as a member of the executive team for the Kempe Center. His portfolio 
includes responsibility for the systems level service delivery research related to child maltreatment 
including a focus on public health, child welfare, and children’s mental health systems. Responsibilities 
include helping to set overall strategic directions for the center, research and evaluation design, research 
management analytic consultation, and oversight of other research and evaluation performed by the 
center. The position is also responsible for specific supervision of grants and contracts as well as 
research and evaluation staff. The work scope includes research and evaluations related to publically 
funded services systems and community-based program evaluation for state social service agencies 
across the country, for the Federal Government, as well as work in other countries. 




JOHN D. FLUKE 


EDUCATION: 

Ph.D., Organizational Decision Science, Union Graduate School, Cincinnati, Ohio, 1995. 

Periodic Graduate course work in Economics (Econometrics), Statistics and Mathematics at the 
University of Colorado in Boulder and Denver, 1981-1990. 

M.A., Anthropology, Pennsylvania State University, University Park, Pennsylvania, 1980. 

B.A., Summa Cum Eaude, Individualized Tutorial Program (ITP), Emphasis in Mathematical 
Anthropology, University of Northern Colorado, Greeley, Colorado, 1976. 


ACADEMIC APPOINTMENTS: 

Graduate School of Social Work, University of Denver, Denver, CO 
Visiting Scholar (2008 to Present) 

In his role as visiting scholar at Graduate School of Social Work Dr. Eluke assists the school to improve 
its approach to developing its child welfare capacity through strategizing with faculty and staff and 
fulfills educational requirements to provide periodic lectures and other substantive content for faculty 
and students. In addition Dr. Eluke represents the school in various capacities through serving on 
external committees and advisory groups. 

Factor Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada 
Status Only Assistant Professor (2012 to present) 

This appointment facilitates coordinated collaboration in the area of research grants, projects and 
publications. This also includes the development of a graduate level course in child welfare decision 
making. 


HOSPITAL, GOVERNMENT OR OTHER PROEESSIONAL 
POSITIONS: 

The American Humane Association, Children's Division, Englewood, CO 
Director of Program Analysis and Research (1979-1999) 

Senior position involved primary responsibility for the analytic consultation, research, and evaluation 
performed by the division. Supervision of the agency's research staff consisting of eight research 
professionals. The work focused on general project design and implementation related to public child 
protective services and community-based program evaluation for state social service agencies across the 
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JOHN D. FLUKE 


country, for the Federal Government and other agencies providing children's services. Dr. Fluke 
directed or contributed to numerous research and program evaluation projects and provided consultation 
to many states and to the federal government in the areas of children’s services decision making, 
administrative data analysis, workload and resource management, and automated child welfare 
information systems. 

Walter R. McDonald & Associates, Inc., Aurora, CO 
Director of Research (1999 to 2003) 

These responsibilities emphasized research design and implementation, capturing and analysis of data, 
and reporting of findings directed toward enhancing the mission of the company. These were achieved 
through the developing and communicating useful information to clients, provision of direction and 
support to the research division, linkage of research and evaluation to communications, marketing, and 
organizational learning. 

Walter R. McDonald & Associates, Inc., Centennial, CO 
Vice President for Research (2003 to 2007) 

As a participant with the management of the company the primary responsibilities were to oversee the 
research and evaluation functions and services of the organization. This position involved oversight for 
the company’s research projects and senior research staff, client relations, and participation in strategic 
planning and implementation. 

Child Protective Services Research Center, American Humane Association, Englewood, CO 
Vice President and Founding Director (2007 to 2012) 

Dr. Fluke rejoined American Humane as the center’s founder and director. Established with private 
funding, the purpose of the research center is to address long-standing issues related to the improvement 
of public child protective services. Research focuses on fundamental issues in child protective services 
and the development of evidence-based policy and practices to effectively address them. Among others, 
the initial areas of focus include assessment and decision-making processes in child welfare cases, 
examination of the system’s racial disparities, improved ways to scale up the implementation of 
evidence-based intervention and treatment practices at agencies and community service providers, and 
the further development of a global network of child abuse and neglect data-collection systems and 
professionals. The Center supports the global network of professionals who form the International 
Society for the Prevention of Child Abuse and Neglect Working Group on National Child Maltreatment 
Data Collection Programs for which Dr. Eluke is the co-chair and a network founder. 

Children’s Innovation Institute, American Humane Association, Englewood, CO 
Vice President (2012) 

In this capacity Dr. Eluke’s research work focused on fundamental services to improve the status of 
children and families including the formation of evidence-based policy and practices. Among others, the 
initial areas of focus include addressing the epidemiology of vulnerable children, the assessment and 
decision-making processes in service delivery, and improved ways to scale up the implementation of 
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JOHN D. FLUKE 


evidence-based intervention and treatment practices at the community level, in the context of the global 
network of systems to address the needs of vulnerable children. 


GRANT AND CONTRACT SUPPORT: 

Total grants/contracts awarded as Principal Investigator or Co-Principal Investigator: $39 million 

Current Grant and Contract Support 

• U.S. Health and Human Services, Children’s Bureau, No Place Like Home: Family Group Decision 
Making for Children and Families Receiving In-Home Services (2011-2014): A federal grant awarded 
to the American Humane Association (AHA), Casey Family Programs (CFP) and three innovative child 
welfare agencies with mature family group decision making (FGDM) programs to test the effectiveness of 
FGDM in safely preventing children from entering or re-entering foster care when they are receiving in- 
home services. As of July 2012 the grant was transferred to the Larimer County Department of Human 
Services with the Kempe Center as the primary operating entity responsible to evaluation and technical 
assistance. The project sites are Larimer County Department of Human Services (LDHS), Colorado; South 
Dakota Department of Social Services (SDDSS); and Texas Department of Family and Protective Services 
(TDFPS). All three sites will participate in a rigorous evaluation (experimental or quasi-experimental), 
longitudinal designs addressing FGDM process, outcomes and cost effectiveness. Dr. Fluke is the Principal 
Investigator, and the grant was funded for $1,400,000 over three years. 

• Chapin Hall, Foster Care Redesign for the Texas Department of Family and Protective Services 
(2010-2012): The Texas Department of Family and Protective Services is in the process of developing a 
modified financial system for reimbursing providers of out of home services in order to leverage funds 
to improve services aimed at creating settings that meet children’s needs and that shorten their time in 
care to planned exits without comprising safety or well being. Under sub-contract to Chapin Hall, Dr. 
Fluke is the Principal Investigator for the Kempe Center for Children. Funding level $90,000 over two 
years. 

• Quality Improvement Center for Differential Response (2008 - 2013): A federal cooperative 
agreement awarded to the American Humane Association (AHA) and transferred to the Kempe Center as of 
July 2012. This project entails 1) development of resources regarding Differential Response (a Child Welfare 
systems level intervention); 2) financial and technical assistance support of child protective services sub¬ 
grantees to evaluate the efficacy of Differential Response being carried out in Colorado, Illinois, and Ohio; 

3) a national evaluation of the three sites; and 4) support for doctoral students engaged in dissertation 
research related to the topic of Differential Response. Dr. Fluke developed and supervised the doctoral 
dissertation award process, and is responsible for aspects of the cross site evaluation. Total support for this 
cooperative agreement is $10,000,000 over 5 years. 

Previously Funded Grant and Contract Support 
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JOHN D. FLUKE 


• Canines and Childhood Cancer: Animal Assisted Therapy in the Context of Pediatric Oncology 
(2011-2012): This grant was awarded to American Humane Association with the objective of 
establishing a multi-site longitudinal random control trial to assess the efficacy of Animal Assisted 
Therapy in improving behavioral health indicators for children with cancer and their caregivers. The 
study encompasses three phases, a developmental phase to synthesize information and work with 
potential hospital based sites, a pilot trial in up to three sites, and a full trial. Dr. Fluke is the Principal 
Investigator for the Pfizer Animal Health grant. Total budget is $3,000,000 for three years. 

• Center for the Study of Social Policy Alliance for Racial Equity in Child Welfare - Research 
Literature Review on Disproportionality and Disparities (2010): This project called for a 
compilation and analysis of research related to racial and ethnic disproportionality and disparities in 
child welfare, primarily beginning with research published since Bob Hill's Research Synthesis in Child 
Welfare: An Update (October 2006). The analysis will also compare studies to previously conducted 
research syntheses by Bob Hill on the topic, as well as any earlier studies that may not have been 
included in previous papers. Dr. Fluke was the Principal Investigator. Funding level was $65,000 for 9 
months. 

• UNICEF, Cross-National Child Disciplinary Practices: An Analysis of the Multiple Indicator 
Cluster Survey (MICS) of Households (2009): Child discipline - deliberate actions on the part of 
parents designed to teach self-control and acceptable behavior on the part of children - has broad 
implications for child well-being. In particular, harsh discipline in the form of corporal punishment (not 
including physical abuse) is associated with adult mental health problems, delinquency, and adult 
criminal behavior. A clearer understanding of disciplinary practices on a global basis has implications 
for the development of effective prevention strategies. This study utilized the Multiple Indicator Cluster 
Survey (MICS3), a representative cross-sectional household survey conducted by UNICEF and its 
partners, and similar data from the Demographic and Health Surveys (DHS) in over 35 countries. Dr. 
Fluke was the Principal Investigator. Funding level was $55,000 for one year. 

• Kingdom of Saudi Arabia National Family Safety Program — Program Report Data Project: 
( 2010 ): The purpose of this project was to review and help expand the analytic capacity of the child 
maltreatment administrative data system under development by the National Family Safety Program 
(NFSP) in the Kingdom of Saudi Arabia (KSA). The scope of work was carried out by team members of 
the Child Protection Research Center, through a subcontract to the International Society for the 
Prevention of Child Abuse and Neglect (ISPCAN). Dr. Fluke was the Principal Investigator. Funding 
level was $40,000 for one year. 

• Texas Department of Family and Protective Services (TDFPS), Child Welfare Disparities 
Initiative Evaluation (2007-2008): This project entailed supporting TDFPS evaluate its initiative to 
identify sources of disparities, develop policy, program, and community responses to address disparities, 
and assess outcomes to determine if outcomes in disparities were impacted. This contract was awarded 
to Walter R. McDonald and Associates, Inc. (WRMA) and entailed support for the analysis of TDFPS 
administrative data. The consultation also extended to implementing the evaluation utilizing the 
Decision Making Ecology framework as a basis for hypotheses. Dr. Eluke was the Principal Investigator 
for WRMA and project funding was for $80,000 for one year. 
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• Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services 
Administration (SAMHSA), Comprehensive Community Mental Health Services for Children and 
Their Families Program, Phase V (2005 - 2007): This 5 year national evaluation involving 25 sites 
was intended to describe the children and families served, including behavioral, emotional, and 
functional characteristics; determine how children and their families change over time; document the 
services delivered and the effects and costs of these services; evaluate how systems of care develop over 
time; investigate the relationship between system change and child and family outcomes; document 
cultural and linguistic competence; examine the nature and effects of family-driven and youth-guided 
care; evaluate the effectiveness of factors influencing the implementation of systems of care and 
evidence-based practices within systems of care; develop evaluation capacity and quality monitoring in 
system of care communities; and identify factors influencing the sustainability of these programs. Dr. 
Fluke was the Principal Investigator for the project. Funding level for this project was in excess of 
$27,000,000 for five years. 

• ORC Macro, Subcontract for the Cross Site Evaluation of the National Child Traumatic Stress 
Initiative of the Substance Abuse and Mental Health Services Administration, Department of 
Health and Human Services (2004 - 2007): This 5 year project entailed the design and implementation 
of a national cross site evaluation for the National Child Traumatic Stress Initiative, a network of over 
50 sites that provides direct and indirect services aimed at supporting the development of trauma 
informed services. WRMA was a subcontractor to ORC Macro, and Dr. Fluke was the WRMA Principal 
Investigator and was the co-leader of the evaluation design team. The funding was $800,000 for five 
years. 

• Office of the Assistant Secretary for Planning and Evaluation, Department of Health and Human 
Services, Secondary Analysis of Data from the National Child Abuse and Neglect Data System 
(2003-2006): A data analysis project to examine three topics of policy concern to related to the 
identification and amelioration of child maltreatment. The topics are male perpetrators, alternative 
response, and a longitudinal analysis. Dr. Fluke is the Principal Investigator for the project. Funding was 
for $350,000 for 3 years. 

• Performance And Outcome Reporting System, Department of Public Social Services, County of 
Riverside California (2002 - 2003): Dr. Fluke was the Principal Investigator on this project to develop 
a performance and outcome reporting system, and was the corporate officer. The project is directed at 
developing a measurement system for services provided by Community Based Agencies under the 
auspices of the Child Abuse Prevention, Intervention, and Treatment (CAPIT) program and the 
Promoting Safe and Stable Families (PSSF) program. Funding was $120,000 for one year. 

• Foster Care Re-Entry Study, Division of Child Welfare Services, Colorado Department of Human 
Services (2001- 2002): An analysis of administrative data to address children who re-enter foster care 
within twelve months of reunification to identify patterns in demographic and service activities that 
might explain re-entry and impact the Adoption and Safe Family Act reunification standards. Dr. Fluke 
was the Principal Investigator responsible for all aspects of this study. Funding was $35,000 for one 
year. 
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• F2F/Internet Training for Data-Based ASFA/MEPA Decision Making, School of Social Work, 
University of Texas at Arlington (2000 - 2003): A three year grant through the Children’s Bureau, 
Administration of Children Youth and Families to develop and test a training curriculum on the 
Adoption and Safe Families Act performance measures for the Texas Department of Protective and 
Regulatory Services. The project entails the development of an internet based training and the use of On 
Line Analytic Processing methods as a basis for the data used in the training. Dr. Fluke was the Principal 
Investigator of the WRMA subcontract funded at $100,000 per year for three years. 

• Texas Department of Protective and Regulatory Services, Texas Title IV-E Adoption Assessment 
Waiver Demonstration Project Cost-Effectiveness Analysis Project, (2001-2003): This project was 
to assess the cost-effectiveness of its title IV-E waiver on adoptions. The project linked with the overall 
evaluation of the project and utilized performance measures and cost data that were derived, as well as 
administrative data resources. Dr. Fluke was the Principal Investigator responsible for all aspects of the 
study and developed an administrative-level computer simulation model to support administrative 
assessment of program efficacy and in forecasting the impacts of changes in services due to the 
introduction of new policy. This was one aspect of an on-going multi-year research project of the 
Department, funded in part by a research grant from the HHS Children’s Bureau. Funding was for 
$250,000 over a two year period. 

• Evaluation Infrastructure Development, Orange County Children’s Commission (2000- 2001): 

The project called for the development of common protocols for evaluation to be employed by 
evaluators of programs funded by the Orange County Children’s Commission. Dr. Fluke was the Project 
Principal Investigator responsible for all aspects of the study. Funding was for $55,000 for a one year 
period. 

• Benefits Workload Study, San Mateo County Human Services Administration (2000-2001): A 

project to assist the county obtain and analyze workload data for purposes of workload standard setting, 
budgeting and workload allocation for CalWORKs/TANF, Medicaid, Food Stamps and other benefits 
programs. Dr. Fluke was the Project Principal Investigator responsible for all aspects of the study. 
Funding was for $200,000 for a one year period. 

• Senate Bill 2030 Workload Study, California Department of Social Services (1999-2000): A 

twelve month statewide project to assist the state obtain and analyze workload data for purposes of 
workload standard setting, budgeting and workload allocation in the area of child welfare services. Dr. 
Fluke was the Project Principal Investigator responsible for all aspects of the study including staffing, 
study design, data analysis and reports. Funding was $1,200,000 for the year. 

• Workload Project; Arizona Department of Economic Security Administration for Children, 

Youth and Families (1998-1999): A six month project to assist the state obtain and analyze workload 
data for purposes of budgeting and workload allocation. Dr. Fluke was the principal investigator. The 
funding was for $175,000. 
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• Colorado Caseload Study; Department of Human Services Division of Child Welfare Services 
(1998-1999): A project to calculate and report on average caseload size for the state and counties to 
address the requirements of Colorado Child Welfare Settlement Agreement to reduce caseloads. The 
project also entails the development of recommendations regarding a workload measurement for child 
welfare in Colorado. Dr. Fluke was the Principal Investigator for AHA for funding of $35,000. 

• Illinois Outcome Measures Project (1997-2000): A project to assist the University of Illinois 
Children’s Research Center develop procedures and analyze administrative data related to the 
production of outcome measures for use in court monitoring. Dr. Fluke was the Principal Investigator 
and the project was funded for $80,000. 

• Colorado Analytic Data Base Development; Department of Human Services Division of Child 
Welfare Services (1997-2000): A project to develop an analytic data base from extracts from the 
Colorado foster care data system and child protective investigation system. The project also involved 
training the Colorado staff in the analytic data base construction methods. Dr. Fluke was the Principal 
Investigator for AHA. Project funding was $35,000. 

• Dynamics of Unsubstantiated Reports: A Multi-State Study (1997-2001): This project was a federal 
research grant to understand unsubstantiated dispositions in a multi-State context. The objectives were to 
develop an understanding of how the characteristics of law, policy, practice, case and decision-making 
are related to the distribution of unsubstantiated dispositions and the outcomes of unsubstantiated 
dispositions. Dr. Fluke was the Principal Investigator for the grant. Funding was for $250,000 per year. 

• Idaho Risk Assessment Project (1997-2000): A three year project to assist the Idaho Department of 
Family and Children Services develop an approach to risk and safety assessment. The project entails the 
development of a risk assessment and safety assessment protocol, instrument, training, and evaluation. 
Dr. Fluke was one of the project Co-Principal Investigators funded by the state for $150,000. 

• Iowa Outcome Measures Project (1997-1999): A project to design an outcome measures data 
collection system for the Iowa Department of Adult, Family, and Children Services. The project will 
address the definition of outcome domains and indicators for Iowa as a whole, the development of 
community outcomes for a selected range of sites, and the development of a system for results based 
budgeting. Dr. Fluke was one of the project Co-Principal Investigators and the funding was for 
$130,000. 

• Casey Managed Care Initiative (1997-2000): This initiative was developed by Annie E. Casey 
Foundation and Casey Family Services to assist children’s services agencies address the issues 
associated with the proliferation of managed care approaches in the field. During the first eighteen 
months the project objectives are to develop a principles document, an outcome measures framework 
and model set of decision making protocols. The American Humane Association is the lead agency in a 
consortium of national organizations including the Institute for Human Services Management and the 
American Bar Association. Dr. Fluke was the Principal Investigator for the $200,000 project. 
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Colorado Subsidized Guardianship Project; Department of Human Services Division of Child 
Welfare Services (1996-1997): The project objectives were to explore models of cost benefit analysis 
in the area of subsidized guardianship policies for kinship care and to develop a computer simulation 
with parameters derived from Colorado foster care data to address cost benefit concerns. The simulation 
model was designed to address a range of policy scenarios and assist the state in developing its 
subsidized guardianship policy. A versions of the Children’s Services Simulation Model was used as a 
base for the simulation and modified to address the project needs. Dr. Fluke was the Principal 
Investigator for AHA for funding of $45,000. 

Innovative Programs with Child Protection Linkages (1995-1997): As a part of the Clark Foundation 
Child Welfare Reform Initiative this project includes an in-depth on-site study of two innovative 
programs: the PATCH model in Cedar Rapids Iowa, and Family Group Decision Making implemented 
in Oregon. The study entails a systematic analysis of the programs, policies, staffing and operations. It 
also includes an ethnographic analysis of four families at each site. Dr. Fluke was the Principal 
Investigator of the evaluation component of the project where the entire project was funded for 
$180,000. 

Philadelphia Outcome Measures Project (1996-2000): This three year project entailed implementing 
an outcome measures data collection system for the City of Philadelphia’s Children and Youth Division. 
Included in the development process are measures of in-home services, short term placement services, 
and long term placement services. Dr. Fluke was one of the project Co-Principal Investigator. Over 
three and a half years the project was funded for approximately $600,000. 

Illinois Child Endangerment Protocol (1994-2000): A project to assist the state of Illinois to develop 
a Child Endangerment Protocol for the Department of Children and Family Services, including 
developing instrumentation for assessing safety, curriculum for child safety, development of a process 
for certifying worker competence in regard to risk and safety and evaluation of the reliability and 
validity of the protocol. Dr. Fluke was the Principal Investigator. The project funding was $325,000. 

Rhode Island Workload Management Project (1994-1995): The project objectives were to develop 
unit of service definitions, case work practice standards, workload standards, define a system to promote 
workload equity and to develop a decision support system to assist DCYF supervisors and 
administrators maintain equitable workloads. Dr. Fluke was the project Principal Investigator for the 
AHA subcontract where the subcontract was for $180,000. 

Kentucky Staffing and Workload Evaluation (1992-1993): The American Humane Association 
assisted the Kentucky Department of Social Services by providing an evaluation of child protective 
service workloads and examining current child welfare practice expectations. Dr. Fluke was Co- 
Principal Investigator of the project and funding was for $55,000. 

Evaluation of the Pennsylvania Approach to Child Protective Service Risk Assessment (1991- 
1993): AHA under contract to Erie County Office of the Children, Youth, and Eamilies and the 
Pennsylvania Office of Children, Youth, and Eamilies conducted several related sub-studies including 
ascertaining the effects of risk assessment implementation on selected aggregate system variables; 
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determining the psychometric properties of three risk assessment models and of the 18 core factors; 
designed a study and implemented the examination of predictive validity; and determined the impact of 
risk assessment implementation on worker and agency operations. Dr. Fluke was the Principal 
Investigator and the project was funded for $230,000. 

• Program Classification and Child Evaluation/Referral Systems for New York City's Foster Care 
Program; American Bar Association Subcontract (1990): This project involved assisting NYC 
develop an information system to match children with appropriate placement resources consistent with 
the terms of Wilder consent decree. Dr. Fluke was the subcontract Principal Investigator and the 
subcontract was for $60,000. 

• Child Protective Services Unit Costing and Service Optimization Project (1990): The purpose of 
this project was to develop a unit costing technology that administrators could use for ongoing analysis 
and financial planning for the Texas Department of Protective and Regulatory Services. The data were 
combined into a software model that prepares information system extract files for organization and 
analysis on a system composed of personal computers. The project included initial development of a 
computer model for optimizing service delivery based on workloads, costs and case outcomes. Dr. 

Fluke was the Principal Investigator with funding of $90,000. 

• Maryland Workload Analysis and Baseline Evaluation Study (1988-1989): The purpose of this 
study was to obtain data on key performance indicators related to the provision of child protective 
services to serve as a baseline for comparison with similar data to be collected after implementation of a 
comprehensive state plan to include risk assessment, a new data system, a worker support system, and a 
competency-based training curriculum. The project also included a workload study to develop CPS 
workload standards for the state. Dr. Fluke was Principal Investigator. This project was funded for 
$40,000 and supplemented by the National Resource Center on Child Protection. 

• Child Protective Services Risk Assessment Reliability Study (1988-1989): The purpose of this 
project was to assess the reliability of a risk assessment instrument developed for use by Colorado CPS 
workers throughout the case process. Dr. Fluke was the Principal Investigator and the project was 
funded for $30,000. 

• Screening in Child Protective Services (1986-1988): This project, performed under sub-contract to 
the American Bar Association, focused on screening and prioritization at intake in five states and twelve 
communities to determine if CPS policy has an effect on actual CPS practice at the point of intake. Data 
to conduct the study were obtained from a national survey of state laws and policies; a survey of 100 
county administrators and supervisors regarding local screening and prioritization; an on-site study of 
CPS screening in 12 communities from 5 states; a survey of representatives from community agencies in 
the 12 participating communities; and case registry follow-up of children who were subjects of CPS 
contacts during the study period. Dr. Fluke was the sub-contract Principal Investigator and the 
American Humane Association sub-contract was for $60,000 per fiscal year. 

• Protective Services Hot-line Feasibility Study; Arizona Department of Economic Security (1986): 

This study entailed a statewide assessment of the need for a statewide hotline, and resulted in a 
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recommendation that it was probably not warranted at the time. Dr. Fluke was the Principal Investigator 
with funding of $90,000. 

Inter-Installation Study of Family Advocacy Central Registry Data Collection Procedures; Office 
of Family Policy and Support, Department of Defense (1989-1990): This project involved 
identifying variations and inconsistencies in data collection procedures. Recommendations were made 
related to improving data collection procedures. Dr. Fluke was the Principal Investigator and funding 
was for $45,000. 

Family Advocacy Central Registry Data Utilization Project; Office of Family Policy and Support, 
Department of Defense (1989-1990): The purpose of this project was to develop training for DOD 
Central registry staff on data utilization. Dr. Fluke was the Principal Investigator and funding was for 
$35,000. 

Ohio Workload Analysis and Resource Management Project (1988-1989): The purpose of this 
project was to define meaningful child welfare service units, to analyze the characteristics of the child 
welfare workload, to expose administrative staff to the unit costing method, and to develop CPS 
workload standards that could be implemented in the four participating Ohio counties. Dr. Fluke was 
Principal Investigator, where funding was $40,000 and supplemented by the National Resource Center 
on Child Protection. 

National Child Abuse and Neglect Reporting Study; National Center on Child Abuse and Neglect; 
Federal Health and Human Services (1979-1988): This multi-year grant was the pioneer effort to 
collect and report on state child abuse and neglect reports. It involved the collection and analysis of 
both aggregate and case level data and was the antecedent of the NCANDS project. Dr. Fluke was 
originally a research analyst and became the Principal Investigator, where the project was funded for 
$250,000 per year. 


OTHER RESEARCH ACTIVITIES: 

Washington State DSHS Children’s Administration, Washington State Workload Study (2006- 
2007): A study to develop recommendations for workload standards for Child Welfare services. The 
study entailed collection of workload data from over 2,000 staff and the development of cost benefit 
information to support the implementation of a State Wide Automated Child Welfare Information 
System. Dr. Fluke was the Strategic Advisor for the project. Funding for the project was $250,000 for 18 
months. 

New York State Office of Children and Family Services, Study of Child Welfare Workload (2006): 

This study concerned the development of workload requirements for child protective services, foster 
care, adoption, and preventive services in the State of New York, including both the public and private 
voluntary sectors. The study addressed the services and tasks conducted by case workers in their case 
management and case planning functions. Dr. Fluke was the Analytic Consultant for this project. 
Funding for the project was $300,000 for 18 months. 
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• American Bar Association (ABA), Division of Children and the Law, Judicial Dependency 
Workloads (2005-2006): WRMA is assisting the ABA in piloting a methodology to measure judicial 
workloads as related to best practice. The process involves collecting time log data from judges; 
supplemental data from trained court observers; Delphi Group data on time estimates; and judicial 
automated systems data. Dr. Fluke researched the development of best practice based workload 
standards forjudges and for estimating workload requirements. Dr. Fluke helped design the study and 
provided consultation and analytic support for this project. Funding for the project was $100,000 for 9 
months. 

• Children’s Bureau, Administration for Children Youth and Families, U.S. Health and Human 
Services, Fourth National Incidence Study of Child Abuse and Neglect (2004-2007): Walter R. 
McDonald & Associates, Inc., is a subcontractor to WESTAT on this project which entails developing 
and implementing a complex sentinel sampling design to measure child maltreatment incidence. Dr. 
Fluke is a member of the team involved with aspects of the research design and is a member of the 
technical advisory group for the project. The funding was for $400,000 for 3 years 

• The Development of a Statewide Approach to the Assessment of Child Safety, Family Strengths 
and Caregiver Protective Capacity, California Department of Social Services (2003): Under sub¬ 
contract to Patricia Schene and Associates, Dr. Fluke assisted the State in developing a uniform 
approach to child and family assessments for the State. The assessments address critical aspects of the 
Child Protective Services re-design effort, including a new system of diversified response. Funding was 
for $40,000 for six months. 

• Aurora Community Mental Health Center, Evaluation of the Aurora Mental Health Center’s 
Grant to Operate a National Child Traumatic Stress Initiative Community Treatment and Service 
Center (NSCTI-CTS) (2002 - 2004): Dr. Fluke has operated as consultant to the Center in developing 
is approach to evaluation and provides evaluation technical assistance to staff who are planning and 
running programs under the auspices of the grant. Funding of $45,000 was for two years. 

• National Study of Child Protective Services Systems and Reform Effort, Office of the Assistant 
Secretary for Planning and Evaluation, Department of Health and Human Services (2000-2003): 

The goal of this study was to provide a comprehensive description of child protective services (CPS) 
systems nationwide and to examine in depth some of the recent innovative reform efforts that are 
underway. Dr. Fluke was the team leader for the data analysis and reporting component. The project was 
funded for $850,000 for a two and half year period. 

• Louis de la Parte Florida Mental Health Institute, Department of Children and Family Studies, 
University of South Florida, Technical Assistance in Developing Child Welfare Performance 
Measurement Reporting (2000-2001). The purpose of this project was to the assist the institute in its 
work the Florida Department of Children and Families to analyze the State’s administrative data and 
prepare an annual report regarding the performance of its foster care system and overall performance of 
its child welfare program. The role was to act as a project advisor and technical assistance consultant. 
Funding was for $20,000 over a one year period. 
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• Texas Screening, Permanency and Decision Making Study (1993-2000): An on-going multi-year 
research project of the Texas Department of Protective and Regulatory services funded in part by a 
research grant from the HHS Children’s Bureau. The goal of the project was to develop decision 
making tools for use by workers and supervisors in reaching decisions about screening reports of child 
maltreatment, providing services, making placements, and closing cases. Dr. Fluke was been an active 
member of the study advisory committee and a member of the research team. Under the auspices of this 
project Dr. Fluke developed the Children’s Services Simulation Model (CSSM) an interactive 
mathematical model designed to assist program management in forecasting the impacts of changes in 
services due to the introduction of new policy. 

• The National Resource Center on Child Abuse and Neglect (1991-1996): The National Resource 
Center on Child Abuse and Neglect (NRCCAN) is operated by the American Humane Association as 
the United States government's center for resources and services to improve the capacity of public and 
private agencies to respond effectively to the problem of child abuse and neglect. Dr. Fluke was a 
project consultant and advisor and was the project manager for the first year. 

• Colorado Family Preservation (1994-1995): The American Humane Association's purpose and 
objective for this project is to help the State ensure that the planning process is as comprehensive as 
possible and meets federal requirements to involve a wide cross-section of the community in the 
development of the five-year plan. Dr. Fluke assisted in developing the evaluation component of the 
plan. 

• Illinois Staff Development (1992-1993): American Humane Association was contracted to develop 
training resources by analyzing the current tasks being performed by child protection investigators and 
then to develop competencies needed to perform the tasks. Dr. Fluke assisted in the development of an 
evaluation of training needs and competencies. 

• Grand Rapids Michigan Children's Service Needs Assessment (1994-1995): The American Humane 
Association conducted a research study of families to address the pertinent demographic characteristics 
of client groups, how these families fall into high, moderate or low risk categories for future 
maltreatment, the specific services needed by the families, and the projected costs for services. Dr. 

Fluke was a consultant to the project. 

• Navy Family Advocacy Program Risk Assessment Consultation (1993): The purpose of the project 
was to assist the Navy to develop a Family Advocacy Program Risk Assessment System. The system 
was designed to be implemented throughout the Navy. Dr. Fluke was a project consultant. 

• National Resource Center for Child Abuse and Neglect (1986-1990): National Center Child Abuse 
and Neglect, Federal Health and Human Services. This was a five year grant to operate a resource 
center to address the systemic needs of service providers and particularly CPS agencies. Dr. Fluke was 
project manager. 
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• Comprehensive Review and Recommendation for Child Protective Services Policy (1985): 

Wyoming Department of Public Assistance and Social Services and South Carolina Department of 
Social Services. These were broad based evaluations of CPS for these states. Dr. Fluke analyzed the 
state policy documents. 

• Analysis of State Agency Variation in Reporting and Disposition; Virginia Department of Social 
Services (1983): The purpose of the project was to evaluate the variation in reporting practices between 
counties. Dr. Fluke performed the analysis of the central registry data. 

• Technical Assistance for the National Child Abuse and Neglect Data System (1991-2000): 

Subcontract to Walter McDonald and Associates. The American Humane Association assists the 
National Center on Child Abuse and Neglect develop, collect and analyze national data on child abuse 
and neglect reports including aggregate reporting data from between 48 and 50 states each year and state 
submitted case level data. Dr. Fluke was the subcontract project manager. 

• Child Welfare Decision Enhancement Project (1993-1995): Texas Department of Protective and 
Regulatory Services. Dr. Fluke was a research consultant to this project. 

• Management Information System Design; Project SAFE Department of Defense (1985-1986): The 

objective of the project was to develop a system approach to data collection across multiple programs. 
Dr. Fluke was a research analyst. 

• Analysis of Family Advocacy Reporting System; Department of the Navy (1983-1984): The Navy 
contracted with American Humane Association to perform the first analysis of its reporting data. Dr. 
Fluke was the research analyst for the project. 

TEACHING: 


Decision Making in Child Welfare: Summer-2013-SWK6503H-F-LEC0101 - University of Toronto, 
Factor Inwentash Faculty of Social Work - 3 Credit Hours (May 13 - 17, 2013): A graduate doctoral 
level course addressing theories, empirical literature, research, and practice in the area of child welfare 
decision making. Research specific to child welfare research topics include approaches and analytic 
techniques that support decision making research and topics such as study design, data construction 
techniques, receiver operator characteristics analysis, structural equation modeling, and multi-level 
modeling. 


PRIVATE CONSULTATION: 

Consultant ( 1997-1998) Workload Measurement for Services to the Developmentally Disabled, State of 
Washington Department of Social and Health Services, Olympia, Washington. 
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Consultant (1995-1998) Expedited Permanency Program Evaluation, Patricia Schene, Eittleton, 
Colorado. 

Consultant (1995) Adoption Yardstick Project, Child Welfare Research Center, University of 
California, Berkeley, California. 

Consultant (1991) California Integrated Service Agency Project, Eourtech Corporation. 

Consultant (1983-1988) Animal Welfare and Control Information System; Metropolitan Animal 
Services Council. 

Consultant (1986) Social Services Reporting System: Adoptions, Sperry Corporation. 

MEMBERSHIP IN PROEESSIONAL ORGANIZATIONS: 

Founding Member, International Society for the Prevention of Child Abuse and Neglect (ISPCAN) 
Working Group on Child Maltreatment Data (ISPCAN-WGCMD). 

Member, International Society for the Prevention of Child Abuse and Neglect. 

Member, American Professional Society on the Abuse of Children. 

Member, Society for Social Work Research 

GOVERNMENT AND COMMUNITY: 

Present 

Member - Expert Advisory Group on the Child and Eamily Service Review Outcome Inidictors, USHHS 
Childrens Bureau, Administration for Children and Eamilies 

Advisory Board Member, Center for Violence and Injury Prevention, George Warren Brown School of 
Social Work, Washington University, St. Eouis, MO, USA. 

Partner, Preventing Violence across the Eifespan Network (PreVAil), partnership of researchers and 
from Canada, United States, United Kingdom, Asia, Europe and Australia, Housed at McMaster 
University, Hamilton, Ontario, Canada. 


Past 

Advisor, Central Independent Advisory Board, Balkan Area Epidemiological Study of Child Abuse and 
Neglect, Institute of Child Health Department of Mental Health and Social Welfare Centre for the Study 
and Prevention of Child Abuse and Neglect, Athens, Greece. 
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Chair, Focal Question Workgroup - Systems, Strategies and Interventions for Sustainable Long-Term 
Care and Protection of Children with a History of Living Outside of Family Care, U.S. Government 
Evidence Summit: Protecting Children Outside of Family Care for the U.S. Agency for International 
Development (USAID), October 2011 - September, 2012. 

Doctoral Committee Member for Dana M. Hollinshead. Heller School for Social Policy and 
Management Brandeis University Successful defense July 2012. 

Technical Advisor to the National Incidence Study on Child Abuse and Neglect. 

Consultation Convener, International Study on Violence against Children, United Nations. 

Technical Advisor on Risk Assessment Research to the Faculty of Social Work, University of Toronto. 

Technical Advisor to the Outcomes Initiative of the New Zealand Children and Young Persons & their 
Families Service. 

Technical Advisor to the Canadian Incidence Study of Reported Child Abuse and Neglect. 

Technical Advisor to the Commission on the Role of Culture and the Assessment of Risk in African 
American Children and Families, Black Administrators in Child Welfare. 

Co-Convener, Roundtable on Child Protective Services Risk Assessment. 

Advisory Board for the National Child Abuse and Neglect Data System. 

Research Advisory Board for the Texas Risk Assessment Evaluation. 

Board Member, Mental Health Service, Inc. 

Board Member and Board President, Aurora Community Mental Health Center. 

Board Member, Arapahoe County Placement Alternatives Commission. 


Other 

Attorney General of Massachusetts. Children’s Rights Lawsuit Connor B. vs. Deval Patrick (2012): 

Dr. Eluke is a scientific expert witness for the defendants in the case brought against the State of 
Massachusetts Department of Children and Eamilies. 

Riggs, Abney, Neal, Turpen, Orbison & Lewis. D.G. vs. Henry (2011): Dr. Eluke is a scientific expert 
witness for the defendants in the case brought against the State of Oklahoma Department of Human 
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Services regarding its foster care system. 


Class Action Lawsuit Alleging Inadequate Child Welfare Services Due to Large Caseloads (1990- 
1991): Legal Services Organization of Indiana. Dr. Fluke was an expert witness. 

REVIEW AND REEEREE WORK: 

Editorial Board, Journal of Technology in Human Services. (2003-present) 
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42. Fluke, J., Peddle NA, Yuan YY, Gaudiosi J and Edwards M (peer reviewed,2002) Charting the Evolution of 

a National Reporting System on Child Maltreatment: A Reflection of 25 Years of Progress. 14* 
International Congress on Child Abuse and Neglect, International Society for the Prevention of 
Child Abuse and Neglect, Denver, CO. 

43. Cherna, M., Harrell, J., & Eluke, J.D. (peer reviewed, 2003). From development to implementation: 

Using data to improve State and local prevention initiatives. Paper presented at the 14* National 
Conference on Child Abuse and Neglect, St. Eouis, MO. 

44. Peddle, N.A., Yuan, Y.T., Gaudiosi, J., Edwards, M.T., & Eluke, J.D. (peer reviewed, 2003). The 

evolution of a National Reporting System on Child Maltreatment: 25 years of progress. Paper 
presented at the 14* National Conference on Child Abuse and Neglect, St. Eouis, MO. 

45. Radel, E., Sedlak, A., & Eluke, J.D. (peer reviewed, 2003). The National Study of CPS Systems and 

Reform Efforts: Results from the Eocal Agency Survey. Paper presented at the 14* National 
Conference on Child Abuse and Neglect, St. Eouis, MO. 

46. Schoech, D. Rycraft, J. Eluke, J.D. and Baumann, D. (peer reviewed 2003).Integrating IT, Research, 

and Performance Measurement: The DEMOS Project. Paper presented at the 2003 Tools That 
Work Conference, Child Welfare Eeague of America, Miami, Elorida. 
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47. Dalgleish, L., Hollows, A., Fluke, J., and Baumann, D., (peer reviewed, 2004) Steps to a Decision 

Making Ecology for Child Protective Services: A Multi-National Research Perspective. 
Presentation at the 15^'’ International Congress on Child Abuse and Neglect, Brisbane, Australia. 

48. Fluke, J., Yuan, Y.T., and Gaudiosi, J., (peer reviewed, 2004) The National Child Abuse and Neglect 

Data System. Presentation at the 15* International Congress on Child Abuse and Neglect, 

Brisbane, Australia. 

49. Gaudiosi, J., Fluke, J.D., Edwards, M., Needell, B., and Sermons, M.W. (peer reviewed, 2004). 

NCANDS in-depth: lessons for program improvement. Presentation at the 2004 Annual 
Conference of the National Resource Center for Information Technology in Child Welfare, Crystal 
City, Virginia. 

50. Schoech, D., Farris, J., Fluke, J.D., and Wittenstrom, K. (peer reviewed, 2004). Putting data where it 

counts: deploying child welfare data through OFAP and data warehouse technology. Presentation 
at the 2004 Annual Conference of the National Resource Center for Information Technology in 
Child Welfare, Crystal City, Virginia. 

51. Gaudiosi, J., Fluke, J.D., Needell, B., and Wulcyzn, F., (peer reviewed, 2005) NCANDS in-depth: 

Fessons on Outcomes. Presentation at the 15* National Conference on Child Abuse and Neglect, 
Boston, MA. 

52. Fluke, J. (peer reviewed, 2006) Mettre a Jour du US National Child Abuse and Neglect Data System 

(NCANDS): Tendances et Comparaisons. Presentation at the 74* Association francophone pour la 
savior (ACFAS), Montreal, Quebec, Canada. 

53. Fluke, J. (peer reviewed, 2006). Symposium on International Child Maltreatment Data Collection 

Programs: Context and Perspectives, The National Child Abuse and Neglect Data System. XVIth 
ISPCAN International Congress on Child Abuse and Neglect, York, England, UK. 

54. Fluke, J., Shusterman, G., Hollinshead, D., Yuan, Y.T. (peer reviewed, 2006) Rereporting and 

Recurrence of Child Maltreatment: Findings from NCANDS. Presentation at the 19th Annual 
Research Conference on Children's Mental Health - A System of Care for Children's Mental 
Health: Expanding the Research Base, Tampa, FF. 

55. Fluke, J.D., Parry C. F., Baumann D., Sheets J., Wittenstrom K., Jeffries V. (peer reviewed, 2006). 

Symposium on Multi-National Research Applications of the Decision Making Ecology (DME) and 
General Assessment and Decision Making (GADM) models to the Issue of Racial and Ethnic 
Overrepresentation of Protective Services: Modeling Decision Factors in the US. XVIth ISPCAN 
International Congress on Child Abuse and Neglect, York, England, UK. 

56. Fluke, J.D., Yuan, Y. T., Gaudiosi, J., (peer reviewed, 2007). Child Neglect - An Overview of 

Incidence, Reporting and Child Characteristics. 20th National Research Conference, Research & 
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Training Center for Children’s Mental Health, USF - Florida Mental Health Institute Tampa, 
Florida 

57. Baumann, D., Capouch, D., Fluke, J., and Ortiz, M. (peer reviewed, 2008) Unlocking Disparities: 

Provider and Community Perspectives. 21st National Research Conference, Research & Training 
Center for Children’s Mental Health, USF - Florida Mental Health Institute, Tampa, Florida. 

58. Fluke, J., Tonmyr, L. (peer reviewed, 2008) International Child Maltreatment Data Collection 

Programs: Context and Perspectives. XVIIth ISPCAN International Congress on Child 
Abuse and Neglect, Hong Kong, China. 

59. Fluke, J., Dalgleish, L., Baumann, D., Wulczyn, F. & Steffen, C. (peer reviewed, 2009) Risk 

Assessment Is Not All There Is: Child Maltreatment Decision Making. 17th National Conference 
on Child Abuse and Neglect, April 2, Atlanta, USA. 

60. James, J., Hargett, B., Fluke, J., Parrish, D., Deserly, K., Miller, O. (peer reviewed workshop, 2009) 

Achieving Service Equity, A Forum on Using Research and Promoting Practice to Address 
Disproportionality. 17th National Conference on Child Abuse and Neglect, April 1, Atlanta, USA. 

61. Gray, J., Fluke, J., Tonmyr, L. (peer reviewed, 2009) The International Society for the Prevention on 

Child Abuse and Neglect (ISPCAN) Working Group on Child Maltreatment Data Collection. 
Congress of the British Association for the Prevention of Child Abuse and Neglect, September 15. 
Swansea, Wales, UK. 

62. Gray, J., Fluke, J., Mansell, J., Madrid, B., Valentine, K., Van Puyenbroeck, B. & Graham, D. (peer 

reviewed, 2009) International Child Maltreatment Data Collection Programs: Context, 
Perspectives, and Prospects for Development. 8^'’ ISPCAN Asian Pacific Regional Conference and 
Australasian Conference, November 17. Perth, WA, Australia. 

63. Casillas, K., Fluke, J., Cappa, C. (peer reviewed, 2010) The multiple indicator cluster survey (MICS) 

of households: cross-national surveillance system to address disciplinary practices. International 
Family Violence and Child Victimization Research Conference, July 13, Portsmouth, NH. 

64. Fluke, J. & Duming, P., (symposium peer reviewed, 2010) Risk Assessment Is Not All There Is: Child 

Maltreatment Decision Making: Translating Science into Service Xllle Congres international 
Education familiale et services pour I'enfance (AIEREE),17-19 novembre 2010, Eirenze, IT. 

65. Gray, J., Eluke, J., Mansell, J., Madrid, B., Valentine, K., Van Puyenbroeck, B. & Graham, D. (peer 

reviewed, 2010) International Child Maltreatment Data Collection Programs: Context, 
Perspectives, and Prospects for Development. 18^’’ ISPCAN Congress, September 26. Honolulu, 

HI. 
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66. Fluke, J., Casillas, K, & Cappa, C., (peer reviewed, 2011) Child Disciplinary Practices: 

Results from the MICS3. 9* ISPCAN Asia Pacific Conference on Child Abuse & Neglect, October 
7, 2011, Delhi, India 

67. Fluke, J., Dalton, E., Needell, B., Wulczyn, F., Courtney, E., Baumann, D., (peer reviewed, 2011) 

Taking research to practice: rethinking outcomes and performance measures for the child and 
family service reviews. Society for Social Work Research Annual Conference, Tampa, EE. 

68. Eluke, J., & Casillas, K., (peer reviewed, 2011) Child Welfare & Children’s Mental Health: Using 

Nationally Representative Data to Identify the Overlap. 24^'’ Annual Children’s Mental Health 
Research and Policy Conference, March 20, 2011, Tampa, Elorida. 

69. Dettlaff, A., Baumann, D., Graham, J. C., Eluke, J. (peer reviewed 2012). Eactors that Influence the 

Removal Decision in Child Protective Services: Development of an Instrument to Understand the 
Decision-Making Process. 2012 Society for Social Work Research Conference, January 13, 
Washington, DC. 

70. Gilbert, R., Eluke, J., O’Donnell, M., Gonzalez-Izquierdo, A., Brownell, M., Gulliver, P., Janson, S., 

& Sidebottom. (Eluke presenter, peer reviewed, 2012). Trends in child maltreatment in six 
developed countries. Society for Social Work and Research: Sixteenth Annual Conference 
Research That Makes A Difference: Advancing Practice and Shaping Public Policy, January 12, 
Washington, DC. 

71. Dettlaff, A. Graham, J. C., Baumann, D., Eluke, J. (peer reviewed 2012). Eactors That Influence the 

Removal Decision In Child Protective Services: Development of An Instrument to Understand the 
Decision-Making Process. Society for Social Work and Research: Sixteenth Annual Conference 
Research That Makes A Difference: Advancing Practice and Shaping Public Policy, January 13, 
Washington, DC. 

72. Eluke, J. (peer reviewed 2012). Racial/Ethnic Disparities in the Child Welfare System: 

Decision Making. The 26th Annual San Diego International Conference on Child and Eamily 
Maltreatment, January 26, San Diego, CA. 

73. Eluke, J., Baumann, D. J. and Detlaff, A. (Eluke & Detlaff, presenters peer reviewed, 2012). Threshold 

Shift: Applications of the Decision-Making Ecology. 18'*’ National Conference on Child Abuse 
and Neglect, April 20, Washington, DC. 

74. Eluke, J., Gilbert, R., O’Donnell, M., Gonzalez-Izquierdo, A., Brownell, M., Gulliver, P., Janson, S., 

& Sidebottom. Eluke, J. (peer reviewed 2012 and symposium organizer). Child maltreatment: 
variation in trends and policies in six developed countries. International Eamily Violence and 
Child Victimization Research Conference, Portsmouth, NH. 

75. Eluke, J. & Duming P. (Symposium organizers, peer reviewed, 2012). An Ecology of Decisions: Child 

Maltreatment and Child Welfare Decision Making, 12th European Scientific Association on 
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Residential & Foster Care for Children and Adolescents, Glasgow, Scotland, September 6. 

76. Graham, J. C., Fluke, J., Baumann, D. J. and Detlaff, A. (Fluke presenter, 2012). The Decision Making 

Ecology of Placing a Child into Foster Care: A Structural Equation Model. 12th European 
Scientific Association on Residential & Eoster Care for Children and Adolescents, Glasgow, 
Scotland, September 6. 

77. Gilbert, R., Eluke, J., O’Donnell, M., Gonzalez-Izquierdo, A., Brownell, M., Gulliver, P., Janson, S., 

& Sidebottom. (Eluke presenter, peer reviewed, 2012). Trends in child maltreatment in six 
developed countries. 12th European Scientific Association on Residential & Eoster Care for 
Children and Adolescents, Glasgow, Scotland, September 7. 

78. Eluke, J. (Eluke presenter and chair, peer reviewed, 2013). Overview and Preliminary Eindings from 

the Evaluation of the National Quality Improvement Center on Differential Response in Child 
Protective Services. 26^^ Annual Children’s Mental Health Research and Policy Conference, 

March 5, 2013, Tampa, Elorida. 

79. Rinehart, R., Maholme, V., Eunice, Ager, A. & Eluke, J. (Ager & Eluke presenter, peer reviewed 

2013). U.S. Government Action Plan on Children in Adversity Strategic framework for 
international assistance 2012 - 2017. ISPCAN European Regional Conference, Dublin, Ireland, 
17th September. 

80. Eluke, J., Merkel-Holguin, E., Hahn, A. & Runyan, D. (Eluke, Merkel-Holguin, Hahn & Runyan 

presenters, peer reviewed 2013). Differential Response: Eooking at this child welfare reform from 
an evidence and rights-based perspective. ISPCAN European Regional Conference, Dublin, 
Ireland, 17th September. 

81. Darnell, A & Eluke, J. (Darnell & Eluke presenter, peer reviewed, 2013). Optimizing Differential 

Response Thresholds in Child Protection: How Much Is Enough? 8'*^ Annual Conference on 
Differential Response, October 22-25, Vail, CO 

82. Darnell, A & Eluke, J. (Darnell & Eluke presenter, peer reviewed, 2014). Optimizing Differential 

Response Thresholds in Child Protection: How Much Is Enough? Society for Social Work and 
Research: Eighteenth Annual Conference Research for Social Change, January 16, San Antonio, 
TX. 


NON-COMPETITIVE 

1. Eluke, J. (Organizer, 1981). Aggregation and Compatibility of Social Service Data. Eirst National 

Conference for Eamily Violence Researchers, Durham, New Hampshire. 

2. Eluke, J. (Organizer, 1984). Policy research in Child Protective Services. Second National Conference 

for Eamily Violence Researchers at the University of New Hampshire. 
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3. Fluke, J. (Organizer, 1984). Working Conference on Child Protective Services Policy in 

conjunction with the 1984 Annual Meeting of the American Association for Protecting Children, 
Anaheim, California. 

4. Fluke, J. (1987). Various Topics, 1987 Annual Meeting of the American Association for Protecting 

Children, Austin, Texas. 

5. Fluke, J. (1988). Various Topics, 1988 Annual Meeting of the American Association for Protecting 

Children, Tacoma, Washington. 

6. Fluke, J., (invited, 1995) The Status of Child Welfare Data. Meeting of the Child Protection Study 

Group, The Institute for Families in Society, University of South Carolina, Wild Dunes, South 
Carolina. 

7. Fluke, J., (Mini-Plenary Organizer and Presenter, (1996) Weaving the Future of Research and 

Evaluation: the Need to Know, Eleventh National Conference on Child Abuse and Neglect, 
Washington, DC. 

8. Eluke, J., (convener and Instructor, 1997) Seminar on the Utilization Computer Simulation in Child 

Protective and Child Welfare Services, Chapin Hall Center for Child and Eamilies, University of 
Chicago, July, Chicago, IE. 

9. Eluke, J., (invited, 1997) Status of the National Child Abuse and Neglect Data System, National Child 

Welfare Conference, Washington, DC. 

10. Eluke, J., (invited, 1997) Subsidized Guardianship: Simulation Model Use and Considerations, 

Kinship Care: A Natural Bridge, August, San Erancisco, CA. 

11. Eluke, J., (invited, 1998) Using Administrative Data to Measure Child Welfare Outcomes. 

Administrative Data In Child Welfare: Research Strategies And Prospects Eor The Euture, June, 
Cornell University, Ithaca, NY. 

12. Eluke, J., (presenter and organizer, 2001), Symposium on the Use of Administrative Data in Child 

Welfare Research and Evaluation. Annual Research Conference: A System of Care for Children's 
Mental Health, Eouis de la Parte Elorida Mental Health Institute, Eebruary, Tampa Elorida. 

13. Eluke, J.D. (invited, 2002). Disproportionate representation of race and ethnicity in child 

maltreatment. Paper presented at the Children’s Bureau Research Roundtable on Children of Color 
in Child Welfare, Washington, DC. 

14. Eluke, J.D. (invited, 2003). Overview of differential response models in the United States: Results 

from the Study of CPS Systems and Reform Efforts. Paper presented at the 4th National Child 
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Welfare Symposium: Community Collaboration and Differential Response, Banff, Alberta, 

Canada. 

15. Fluke, J. D., (invited, 2004) Managing the Risk of Harm to Children, presentation to the meeting 

“Keeping Children Safe: The Child Protection System Challenge?” Wingspread Conference 
November 10-12, 2004 

16. Fluke, J.D., (invited, 2007). The National Child Abuse and Neglect Data System (NCANDS): 

development and utility of an existing system for collecting data and monitoring child abuse in the 
US. EUROPEAN SEMINAR ON MONITORING SYSTEMS OE CHIED ABUSE Child abuse: 
which kind of data for monitoring? 18 January 2007, Istituto degli Innocenti, Elorence, Italy. 

17. Eluke, J. (invited, 2009) Outcomes in the Eong Run: Ingredients for Children’s Protection and Welfare 

Outcomes Programs. The Second Canadian Roundtable on Child Welfare Outcomes Center for 
Research on Children and Eamilies, 8-9 October, 2009. Centres of Excellence for Children’s 
Well-Being: Child Welfare, Montreal, Canada. 

18. Eluke, J., (invited, 2010) Safety outcomes and safety decision policy: NCANDS and administrative 

data capacity, limitations, and prospects. Partners for Our Children, November 2, Tacoma, WA 

19. Eluke, J. (invited, 2011). Eocal Question 3: Effective sustainable long-term care and protection of 

children with a history of living outside of family care: Presentation of the Evidence. U.S. 
Government Evidence Summit: Protecting Children Outside of Eamily Care, December 13, 
Washington, DC. 

20. Eluke, J. (invited, 2011). Child Maltreatment Eatalities: Purpose, Measurement, Analysis, Decisions. 

Child Safety Eorum, Casey Eamily Programs, November 10, Washington, DC. 

21. Eluke, J., (invited, 2011). Child Protection and Systems of Intervention: Perspectives on Cross System 

Data Comparison. CIS-2008 Provincial / Eirst Nations Research Network Workshop, October 21, 
2011, Toronto, Canada. 

22. Eluke, J., (invited, 2011) Indicators and the protection of children victims of exploitation and violence. 

Stakeholders Meeting “Applying the European Union Agency for Eundamental Rights Indicators 
on the Rights of the Child: Policy Priorities for Data Collection.” European Union Agency for 
Eundamental Rights, Eebruary 22-23, Vienna, Austria. 

23. Eluke, J., (invited, 2011) Research synthesis on child welfare disproportionality and disparities: child 

welfare entries. Race & Child Welfare: Disproportionality, Disparity, Discrimination: Re- 
Assessing the Eacts, Re-Thinking the Policy Options -Working Conference, January 28-29, 
Harvard Eaw School, Cambridge, MA. 

24. Eluke, J. (invited 2012). A cross-national view of child protective systems. Child Maltreatment 

Research, Policy and Practice for the Next Generation: A Workshop Convened by the Board on 
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Children, Youth, and Families, Institute of Medicine and National Research Council of the 
National Academy of Sciences, January 31, Washington, DC. 

25. Costello, T., Fluke, J., Casillas, K., Baumann, D., Fuller, T., Freitag, R., Edwards, M. (invited 2012). 

Decision Making in Child Welfare: A Discussion of Tools, Influences, Outcomes, and Research. 
18* National Conference on Child Abuse and Neglect, April 18, Washington, DC. 

26. Fluke, J. (invited 2012). Steps to a Decision Making Ecology: Implications for Child Welfare 

Prevention and Intervention. 2012 Council on Contemporary Eamilies Annual Conference, April 
27, Chicago, IE. 

27. Eluke, J. (invited 2013). Decisions and Disparities: Disentangling Sources of Inequity. Shubert Center 

for Child Studies, Case Western Reserve University, Eebruary 7, Cleveland, OH. 


28. Eluke, J. (invited 2013). Decision Making Ecology: The Continuum of Child Welfare Decisions. 

Decision-making on child care symposium: Erom A to Z in decision-making in child care, 
Groningen, Netherlands, 20th September 2013. 

29. Eluke, J. (invited 2013). Rethinking the Eunctioning of Child Protective Services. “On the Erontline” 

Initiative Meeting, Annie E. Casey Eoundation, Baltimore 15 October, 2013. 

30. Eluke, J. (invited 2014). Perspectives on Risk and Integrated Data in Decision Making Contexts: Child 

Maltreatment Surveillance, Prevention, and Intervention. Expert Meeting Erontiers of Risk 
Epidemiology in Early Childhood, German Youth Institute, Munich, Germany, January 29*, 2014. 

31. Eluke, J. (invited 2014). Eifespan Data Einkage and Integration: Benefits for Preventing Harm to 

Children. National Family Safety Program, National Guard Health Affairs, Riyadh, KSA, Eebruary 
18,2014 

32. Eluke, J. (invited 2014). A Critical Reflection on Decision Making Contexts and the Potential Role of 

Predictive Risk Models. Advancing the Science of Children’s Services Through Earge Data, 
University of Southern California, School of Social Work, Eos Angeles, CA, Eebruary 28, 2014. 

33. Eluke, J. (invited 2014). Decision Making Ecology and Applications. Decision Making Seminar, 

Haruv Institute, Hebrew University, Jerusalem, Israel, 3 March, 2014 


PEER REVIEWED PUBLICATIONS IN REVIEW (1): 

1. Darnell, A., Eluke, J., Casillas, K., Rudlang-Perman, K., and Guterman, K. (in review). Exploring Risk 
Thresholds for the Alternative Response Track in Differential Response Systems: A County-Eevel 
Analysis of Utilization of the Alternative Response Track and Re-Reporting Rates. Child Abuse & 
Neglect. 
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DANA M. HOLLINSH EAD, Ph.D. 

1 • dana.hollin$head@childrenscolorado.org 

CURRICULUM VITAE 

Education 

Ph.D., 2012 Social Policy, Children, Youth, and Families Policy concentration. 

Heller School of Social Policy and Management, Brandeis University, Waltham, MA. 

Dissertation: Do Outcomes Reflect Performance, Demographic Dividends, or Policy Artifacts? 

A Study of the Effects of Child Welfare Policies, Practices, and Demographics on State Child 
Maltreatment Recurrence Rates. 

Committee: Jeffrey Prottas (Chair), John Fluke, Andrew Hahn, and Natalia Sarkisian 

2002- 2003: Recipient of the Sol Chaikin Fellowship. 

2003- 2004: Recipient of the Shapiro Fellowship. 

2004: Teacher’s Assistant for Professor Anita Hill’s course on Race and the Law. 

M.P.A., 1995 Policy Analysis and Program Evaluation emphasis. Child and Family Studies concentration. 

Trachtenburg School of Public Policy and Public Administration, George Washington University, 
Washington, D.C. 

M,A., 1995 Public Policy, Women's Studies concentration. 

George Washington University, Washington, D.C. 

B.A., 1989 Psychology. 

Colby College, Waterville, ME. 

Skill Summary 

• Extensive child welfare research, program evaluation, and policy analysis experience utilizing advanced 
quantitative and qualitative data collection, analysis, report writing, and project management skills. 

• Insight into federal, state, and local perspectives on child welfare programs, policies, practices, administrative 
data, performance measurement, program improvement, and system reform efforts. 

• Demonstrated presentation, training, technical assistance, publication, grant writing and grant reviewing 
experience. 

Professional Experience 

Assistant Research Professor, July 2013 - present 

Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, Denver, CO. 

Child and Family Services Review and Title IV-E Review Consultant Reviewer, August 2002 - August 2008 
Johnson, Bassin & Shaw, Silver Spring, MD. 

Child Welfare Program Specialist, January 2001 - August 2002 

United States Department of Health and Human Services, Administration for Children and Families, Boston, MA. 

Senior Research, Program, and Policy Analyst, September 2000 - December 2000 
Program, Policy, and Research Analyst, September 1997 - August 2000. 

American Humane Association, Children’s Division, Englewood, CO. 

Research and Policy Associate, April 1996 - August 1997 
Center for the Study of Social Policy, Washington, DC. 

Research Assistant Intern, October 1994 - April 1996 
Child Welfare Eeague of America, Washington, DC. 



Primary Research Interests 


• Child protective services research, policy analysis, program evaluation, and system reform efforts. 

• Influence of policies, practices, and demographics on child welfare decisions and outcome measures. 

Peer Reviewed Journal Article 

Fluke, J., Shusterman, G., Hollinshead, D., & Yuan, Y.Y. (2008). Longitudinal Analysis of Repeated Child 
Abuse Reporting and Victimization: Multistate Analysis of Associated Factors, Child Maltreatment, 13, 76-88. 

Book Chapter 

Hollinshead, D., & Fluke, J. (2001). What Works in Safety and Risk Assessment for Child Protective Services. 

In M. Kluger, G. Alexander & P. Curtis (Eds.), What Works in Child Welfare. (67-92). Washington, DC: CWLA. 

Monographs and Technical Reports 

National Quality Improvement Center on Differential Response in Child Protective Services. (2014). Final 
report: QIC-DR cross-site evaluation. (Contributing Author) 

Fluke, J., Shusterman, G., Hollinshead, D., & Yuan, Y.Y. (2005). Longitudinal Analysis of Repeated Child 
Abuse Reporting and Victimization: Issue Brief #3. Washington, DC: United States Department of Flealth and 
Fluman Services (US DFIFIS), Office of the Assistant Secretary for Planning and Evaluation (OASPE). 

Shusterman, G., Fluke, J., Hollinshead, D,, & Yuan, Y.Y. (2005). Alternative Responses to Child Maltreatment: 
Findings from NCANDS: Issue Brief #2. Washington, DC: US DFIHS, OASPE. 

Fluke, J., & Hollinshead, D, (2003). Child Maltreatment Recurrence. Duluth, GA: US DFIFIS National Resource 
Center on Child Maltreatment. 

Fluke, J., Parry, C., Shapiro, P., Hollinshead, D., & Bollenbacher, V. (2002, September). The Case on 
Unsubstantiated. Policy & Practice. Washington, DC: APFISA. 

Fluke, J., Parry, C., Shapiro, P., Hollinshead, D., Bollenbacher, V., Baumann, D., & Davis-Brown, K. (2001). 
Dynamics of Unsubstantiated Reports of Child Abuse and Neglect: A Multi-State Study. Englewood, CO: 
American Flumane Association, American Public Human Services Association, and Walter R. McDonald and 
Associates. 

American Humane Association. (2000). Assessment of the District of Columbia Bifurcated Intake and 
Investigation System. Englewood, CO: AHA. (Primary Author) 

American Humane Association and Walter R. McDonald and Associates. (2000). California Department of Social 
Services S.B. 2030 Child Welfare Services Workload Study. Englewood, CO: AHA. (Contributing Author) 

American Humane Association. (1999). Developing and Implementing an Outcome-Based Performance 
Monitoring System for Child Welfare Services in Iowa: Final Report. Englewood, CO: AHA. (Contributing 
Author) 

American Humane Association. (1998). Developing and Implementing an Outcome-Based Performance 
Monitoring System for Child Welfare Services in Iowa: Results of the Evaluability Assessment. Englewood, CO: 
AHA. (Primary Author) 
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American Humane Association. (1998). The Research Basis for Risk Assessment. In Idaho Family and Children 
Services Risk Assessment Training Handbook. Englewood, CO: AHA. (Primary Author) 

American Humane Association, University of Southern Maine Muskie Institute, and Child Welfare Policy and 
Practice Group. (1998). A Comprehensive Review of the Cuyahoga County Department of Children and Family 
Services. Englewood, CO: AHA. (Contributing Author) 

Center for the Study of Social Policy. (1997). 1996 Assessment of the Progress of the Arkansas DHS/DCFS in 
Meeting the Requirements of the Ansela R. Settlement Agreement, Volume I: Case Record Review and Volume IF 
Site Visit Report. Washington, DC: CSSP. (Contributing Author) 

Hollinshead, D, (Ed.) (1997, Spring). SafeKeeping: A Newsletter for the Edna McConnell Clark Foundation’s 
Community Partnerships for Protecting Children. Washington, DC: CSSP. 

Center for the Study of Social Policy. (1996). Profiles of Missouri’s Community Partnerships and Caring 
Communities. Washington, DC: CSSP. (Contributing Author) 

Presentations and Training 

Hollinshead, D, (2007). The Ecology of Child Maltreatment Recurrence: A Study of Relationships between 
Policies, Practices, Demographics and Outcomes. Presentation at the 16* National Conference of Child Abuse 
and Neglect, Portland, OR. 

Fluke, J., & Hollinshead, D, (2003). What Do We Know About Child Maltreatment Recurrence! Presentation at 
the 14* National Conference on Child Abuse and Neglect, St. Eouis, MO. 

Hollinshead, D., Melendez V., & Munro, J. (2002). The Role of the Courts in Supporting State Adherence to Title 
IV-E and IV-B Laws and Regulations. Presentation at the Maine District Court Judge’s Conference, Sunday River, 
ME. 

Fluke, J., Parry, C., & Hollinshead, D, (2001). Dynamics of Unsubstantiated Reports of Child Maltreatment. 
Presentation at the 13* National Conference of Child Abuse and Neglect, Albuquerque, N.M. 

Melendez, V., & Hollinshead, D, (2001). ASF A and the Child and Family Services Reviews. Presentation to the 
Maine State Eegislative Standing Committee on Health and Human Services, Augusta, ME. 

Squadrito, E., Melendez, V., Dobbyn, B., Hollinshead, D., & Munro, J. (2001). Preparing to Conduct the CFSR 
Onsite Review. Presentation at the Massachusetts CFSR State Team Training. 

Hollinshead, D, (2000). History and Overview of Risk and Safety Assessment. Presentation at the 14* National 
Roundtable on CPS Risk Assessment, San Francisco, CA. 

Parry, C., & Hollinshead, D, (2000). The Dynamics of Unsubstantiated Reports: Policy Related to Safety and 
Risk. Presentation at the 14* National Roundtable on Risk Assessment, San Francisco, CA. 

Berdie, J., Alexander, S., Salovitz, B., & Hollinshead, D. (2000). Idaho Risk Assessment: Cross-State On Site 
Training. 
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McDaniel, N., & Hollinshead, D, (1998). The Adoption and Safe Families Act and the Proposed Child and 
Family Services Program Reviews. Presentation at the 12* National Conference on Child Abuse and Neglect, 
Cincinnati, OH. 

Policy Analyses and Advocacy Efforts 

American Humane Association. (1999). Comments on the Proposed Adoption and Safe Families Act Outcome 
Measures. Englewood, CO: AHA. (Primary Author) 

American Humane Association. (1998). Comments on the Proposed Regulations for the Child and Family 
Services Reviews. Englewood, CO: AHA. (Primary Author) 

American Humane Association. (1998). Comments on the Proposed Regulations for the Adoption and Safe 
Families Act. Englewood, CO: AHA. (Primary Author) 

Hollinshead, D, (1998). Alcohol and Other Drug Abuse and the Termination of Parental Rights: A Challenge for 
Implementation of the Adoption and Safe Families Act of 1997. Protecting Children, 14(f>), 15-20. 

Hollinshead, D, (1998). Domestic Violence and ASFA: Preventing the Double Victimization of Abused Mothers. 
Protecting Children, 14(3), 19. 

Child Welfare Research, Program Evaluation, Policy Analysis, And Training Projects 
Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

• Quality Improvement Center on Differential Response in Child Protective Services Cross-Site Evaluation. 
Conducted data analysis and edited and wrote reports for a project examining the processes and outcomes of a 
randomized control study of three sites’ differential response programs. 

• No Place Like Home Family Group Decision Making Evaluation. Conducted data analysis and edited and 
wrote reports for a project examining the processes and outcomes of a mixed methods study regarding the 
implementation, utilization and comparative outcomes in three sites employing family group decision making. 

• Saudi Arabia Project. Conducting data analysis to examine the association between adverse childhood 
experience and long-term health outcomes using ACES survey data collected across Saudi Arabia. 

US DHHS Administration for Children and Families Program Reviews: 

• Child and Family Services Reviews. Served as a site co-lead, federal reviewer, or consultant reviewer on 10 
CFSRs. 

• Title IV-E Eligibility Reviews. Served as a federal or consultant reviewer on four IV-E reviews. 

• Adoption and Foster Care Analysis and Reporting System (AFCARS) Reviews. Served as federal reviewer 
on three AFCARS reviews. 

• State Automated Child Welfare Information System (SACWIS) Reviews. Served as federal reviewer on one 
SACWIS review. 

American Humane Association projects: 

• Assessment of the District of Columbia Bifurcated Intake and Investigation System. Project Manager, 
Managed an assessment of the feasibility of unifying intake and investigation responsibilities. Reviewed 
policies and evaluated cooperative agreement between the District of Columbia Metropolitan Police 
Department and the D.C. Child and Family Services Agency (CFSA). Created interview and focus group 
protocols, analyzed data, monitored project budget, and functioned as principal liaison between AHA, project 
consultants and the client. Conducted data analysis using multiple quantitative and qualitative data sources. 
Conducted comparative analysis of alternative models of intake and investigation for CFSA. Synthesized 
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information from all sources to develop recommendations, write and edit final report. 

Dynamics of Unsubstantiated Reports of Child Abuse and Neglect. Reviewed state policies, revised and 
implemented analytic plan, synthesized national administrative data (NCANDS Summary Data Component 
(SDC), and Detailed Case Data Component (DCDC) data) with policy review data, coded and conducted 
quantitative and qualitative data analysis for a Federally-funded 50 state study which aimed to explain 
variance in unsubstantiation disposition rates across the states. Wrote and edited final policy analysis report. 
Assisted with project planning and management. 

California Department of Social Services Workload Study. Conducted focus groups to develop workload 
time study codes and definitions, define work parameters for special county initiatives, and to set minimal and 
optimal standards for child welfare staff workloads for a study aimed at updating the State’s child welfare 
services budgeting methodology. Developed training materials and coordinated logistics for statewide data 
collection train-the-trainers training, involving over 500 staff across the State. Provided technical assistance to 
study participants, analyzed qualitative data, synthesized findings, drafted report recommendations, and co¬ 
wrote and edited final report. 

Comprehensive Review of the Cuyahoga County, Ohio Department of Children and Family Services. 
Developed, pilot-tested, and trained reviewers on case record review (CRR) instrument and process which 
was part of a comprehensive child welfare system performance review. Conducted CRR and lead CRR 
quality assurance effort. Conducted interviews, focus groups, and other data analysis using multiple 
quantitative and qualitative data sources, synthesized findings and generated program and system reform 
recommendations, co-wrote and edited final report. 

National Child Abuse and Neglect Data System (NCANDS), Technical Team Member. Assisted with the 
provision of technical assistance to the Administration for Children and Families’ Region IV states in 
validating, clarifying, and interpreting their SDC and/or DCDC data, which are voluntarily submitted by 
states to the Federal government and reflect child protective services’ intake, investigation and other related 
data. Analyzed and prepared data and edited state co mm ents for publication in Child Maltreatment 1999. 

Arizona Administration of Children and Family Services Workload Study. Co-developed workload study 
template, provided technical assistance to study participants, conducted quantitative data analysis, and 
prepared reports for a statewide study of child welfare staff workloads. Co-wrote final report. 

Hennepin County, MN, Child Welfare System Reform Evaluability Assessment. Project Manager. Created 
protocols for and conducted focus groups and interviews to identify program and policy issues of concern and 
determine the feasibility of and plans for further evaluation for a project aimed at assessing agency 
performance and developing recommendations for system and program reform. Synthesized, analyzed and 
reported findings to develop framework for subsequent program evaluation. 

Iowa Division of Children and Family Services Outcome-Based Performance Monitoring System Project. 
Co-facilitated focus group charged with identifying and selecting outcome measures in a project that 
developed a statewide, outcome-based, performance monitoring system. Wrote project reports. 

Idaho Risk and Safety Assessment Training. Conducted literature review on the empirical basis for safety 
and risk assessment and individual risk and safety factors, assisted in the development of standardized State 
risk and safety assessment instruments, contributed to development and revision of training curricula, and 
delivered pilot and roll-out training to social workers and supervisors across the state. 

AHA Roundtables. Assisted in the development of national roundtable themes and agendas. Reviewed 
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presentation abstracts, selected speakers, facilitated workshops for, and delivered presentations at AHA’s 
Outcome Measures, Risk Assessment, and ASFA Roundtables. 

• AHA ASFA and CFSR Advocacy efforts. Spearheaded agency ASFA and CFSR advocacy initiative. 
Authored agency position papers on the proposed ASFA regulations, outcome measures, and CFSRs. 
Authored articles about the potential impact of ASFA on both substance abuse and domestic violence-affected 
families in the child welfare system. Facilitated ASFA roundtable workshops. Co-facilitated a think tank on 
ASFA and the CFSR at the 12* National Conference on Child Abuse and Neglect. 

Center for the Study of Social Policy Projects: 

• Arkansas Division of Child and Family Services Case Record Review and Site Visits. Reviewed and co¬ 
supervised review of a representative, stratified sample of child welfare services case records to assess staff 
and system performance. Conducted interviews of and focus groups with child welfare and privatized CPS 
system personnel. Cleaned and analyzed quantitative data on over 1,200 cases from across the state. Wrote 
and edited reports. 

• District of Columbia Department of Children and Family Services Child Fatality Review. Reviewed child 
fatality case records to identify practice concerns and assess staff compliance with child fatality review 
protocols. Identified strengths of and challenges to system performance. 

• Community-Based Child Protective Services Evaluation Workgroup. Conducted literature review and 
guided workgroup discussion regarding options for inter- and intra-site evaluations of a four-state, 
community-based, foundation-supported child protective services initiative. 

• Missouri Caring Communities Case Study, Created interview protocols and conducted interviews of 
program personnel and community members for an assessment of a school-linked community services 
initiative spanning the state. 

• SafeKeeping Newsletter, Editor. Wrote and edited articles for newsletter aimed at diverse audience (child 
welfare administrators, staff, clients, researchers, and community members) focusing on community-based 
approaches to child protective services. Managed production and distribution tasks. 

• Clearinghouse on Community-Based Approaches to Child Protective Services. Developed structure of and 
collected resources for web-based clearinghouse. Managed dissemination of clearinghouse information to the 
public. 

Honors 

• Selected Participant, 2000 Summer Research Institute, Cornell University, Sponsored by the National Data 
Archive on Child Abuse and Neglect. 

• Member, Pi Alpha Alpha Flonorary Society, 1995. 

Service Activities 

• Volunteer Ski Coach, New England Disabled Sports, Loon Mountain, Lincoln, NFl. 2005-2008. 

• Elected Class Representative, Fleller School of Social Policy Ph.D. Committee. 2002-2003 and 2003-2004. 

• Member, Thirteenth National Conference on Child Abuse and Neglect Advisory Committee, 2000. 

Computer Skills 

Proficient in MS Word, Excel, PowerPoint, and Project; SPSS; and STATA. 
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Nalty Curriculum Vitae 1 


Matthew J. Nalty 

Education: 

2010 Metropolitan State College of Denver 

Baehelors of Arts in Psyehology 

Professional Experience: 

May 2012 - Present Kempe Center for the Treatment and Prevention of Abuse and Neglect 

Professional Researeh Assistant - Department of Pediatrics, University of Colorado 
Supervisors: Katherine Casillas, Ph.D. & John Fluke, Ph.D. 

Duties: 

• Consulting Evaluator for Texas Foster Care Redesign. Advise the Texas 
Department of Family and Protective Serviees (TDFPS) on performanee measures 
as the foster care system transfers to private vendors. Utilize Geographie 
Information Systems to analyze bed availability by level of eare within a speeified 
region and distanee traveled when a ehild is removed from home and plaeed in state 
paid foster care. Mapping produets and reports ineluding statistieal analysis are 
ereated to inform TDFPS on baselines, prospeetive targets, and aetuals as provided 
by the regional vendor. 

• Evaluator for Family Conneetions Grant: No Plaee Fike Home. Survey 
eonstruction, data eolleetion, and outeome analysis. This study is focused on 
Family Group Deeision Making (FGDM) as it relates to families eurrently 
receiving in-home services. Datasets are eonstructed from earegiver, faeilitator, and 
meeting fidelity surveys then merged with administrative data extraeted from State 
SCAWISs. This data informs semi-annual and annual reports to the Federal Projeet 
Offieer as well as presentations at the National Conferenee on Child Abuse and 
Negleet on the suecesses and challenges of implementing FGDM as a best praetiee. 

• Teehnieal Speeialist for Colorado Department of Human Serviees Training 
Academy. Provide teehnieal support for all aspects of the Aeademy as an employee 
of the CMO. Setup and maintain Regional Training Centers in Garfield, Fremont, 
and Farimer eounties. 

• Provide analytie support to a variety of projeets throughout the Kempe Center. All 
evaluation and analytie duties require an advaneed level of experienee with 
Mierosoft produets (e.g.. Word, Exeel, and SharePoint) and SPSS. 

June 2010 - May 2012 American Humane Association 

Research Assistant & IRB Administrator - Children’s Innovation Institute 
Supervisors: Katherine Casillas, Ph.D. & John Fluke, Ph.D. 

Duties: 

• Provide technical and analytic support to programs across all programmatic areas; 
data collection, preparation, construction, and interpretation; prepare and 
disseminate research reports and information to internal staff, external clients and the 


13123 E 16‘^ Ave., B390 
Aurora, CO 80045 
Matthew.nalty@ucdenver.edu 
303-864-5132 



Nalty Curriculum Vitae 2 

general publie in mediums that are easily understood by both expert and layperson. 
Use statistieal and database software (e.g., SPSS, Aeeess) to enter, proeess, and 
analyze datasets; projeet management; teehnieal assistanee and support related to 
Humane Researeh; and some qualitative researeh, evaluation and analysis, 
ineluding interviews and focus groups. 

• Serve as an IRB Administrator; main point of eontaet for principal investigators, 
review ineoming applieations, faeilitate full review board meetings, create and 
modify applieation and approval/rejection forms. 

• Manage Geographic Information Systems (GIS) with primary responsibility for 
designing, developing, implementing, and managing eomputer based GIS 
databases and eartographie information. Worked with the state of Texas to 
evaluate out of home plaeement capaeity and distanee of plaeement from a ehild’s 
home eounty. Using data from the National Child Abuse and Negleet Data System 
evaluated frequency of reports and substantiated reports at loeal state and national 
levels. Created numerous map produets that display programmatie impaet used in 
response to RFPs/RFIs and other funding streams. 

• Manage Mierosoft SharePoint System for all departments. This ineludes site 
eonstruetion, user management, doeument sharing management, training on 
SharePoint navigation for internal staff and external partners with an emphasis on 
building team sites, managing permissions and lieenses for site eolleetions, design 
ehanges, and enhaneing user experienee. 


Selected Publications & Presentations: 

Nalty, M, J., & Fluke, J. (2013, June). Children in out-of-home placement: How far have they moved? 

Oral paper presentation at ESRI International User Conferenee, San Diego, CA. 

Tull, T., & Nalty, M, J. (2011). Women’s bean projeet: From a hill of beans to a produetive life 

[Monograph].Cownc// for Standards in Human Services Education, Monograph Series, 160-189. 
http://www.cshse.org/pdfs/Hagen_8-4-2011 .pdf 

Andresen, D. R., & Nalty, M, J. (2010, May). Rotation of currently-viewed objects updates representations of 
previously-viewed objects. Poster session presented at Assoeiation for Psychologieal Seience, Boston, 
MA. 

Nalty, M, J., & Andresen, D. R. (2010, April). Dynamic versus static rotation cues for updating visual object 
representations. Oral presentation at Rooky Mountain Psychology Association, Denver, CO 

Hatch, J., King, S., Nalty, M,J., Ung, N., Valdez, G. (2010, April). Does anonymity affect extroversion? Poster 
presentation at Rooky Mountain Psyohology Assoeiation, Denver, CO 

Nalty, M. J,, & Andresen, D. R. (2009, April). Object recognition from different viewpoints. Oral 
presentation at Rooky Mountain Psyohology Assoeiation, Albuquerque, NM. 

Abeyta, L., Brown, A., King, S., Nalty, M. J., Reify, B., & Valdez, G., et al. (2009, April). The effects of 

aggression on conformity. Poster presented at Rooky Mountain Psyohology Assoeiation, Albuquerque, 
NM. 
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Provide the following information for the Senior/key personnel and other significant contributors. 
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NAME 

POSITION TITLE 

Fred H. Wuiczyn 

Senior Research Fellow 

eRA COMMONS USER NAME (credential, e.g., agency login) 



EDUCATION/TRAINING (Begin with baccaiaureate or other initiaiprofessional education, such as nursing, inciude postdoctoral training and 
residency training if applicable.) 


INSTITUTION AND LOCATION 

DEGREE 
(If applicable) 

MM/YY 

FIELD OF STUDY 

Juniata College 

BS 

June, 1975 

Psychology/Sociology 

Marywood University 

MSW 

May, 1979 

Social Work 

University of Chicago 

Ph.D. 

Dec., 1986 

Social Policy 


A. Personal Statement 

Throughout my career, I have been interested in bridging that gap between research, policy, and practice. To 
that end, I have dedicated significant time to the design of sophisticated longitudinal data based from the 
electronic records collected by public agencies. This work led to the development of the Multistate Foster Care 
Data Archive. With the records of more than 3 million covering in some cases 25 years, the Archive is the 
largest and oldest repository of longitudinal placement data available anywhere in the world. Among other 
research projects carried out with that data, we have developed a sophisticate simulation laboratory for 
studying the likely impact of policy and program on the well-being of children touched by the child welfare 
system. 


B. Positions 


Positions and Empioyment 


1979-1982 

1982- 1985 

1983- 1984 

1985- 1986 

1986- 1996 
1989-1990 
1992-1997 
1994-1996 


Research Assistant, Children's Policy Research Project, School of Social 
Service Administration, University of Chicago 

Research Associate, NORC, and School of Social Service Administration, 
University of Chicago 

Project Director, Young Parent Program Evaluation, Social Policy Research 
Center, NORC and the University of Chicago 

Director, Information Resources, Chapin Hall Center for Children at the 
University of Chicago 

Senior Poiicy Anaiyst, New York Department of Social Services 
Visiting Research Professor, State University of New York at Albany 
Assistant Professor, Columbia University School of Social Work 
Director, Child Welfare Managed Care Initiative, NY Department of 
Social Services 


1996-2002 

1996-2010 

2002-2006 

2004-Present 


Research Associate (Assistant Professor), University of Chicago 
Research Feiiow, Chapin Hall Center for Children at the University of 
Chicago 

Research Associate (Associate Professor), University of Chicago 
Director, Center for State Foster Care and Adoption Data, Chapin Hall 
Center for Children at the University of Chicago 














2006-Present Research Associate (Professor), University of Chicago 

2010- Present Special Advisor, Commissioner Bryan Samuels, Administration for 

Children, Youth and Families, US Department of Health and Human 
Services (as an I PA) 

2011- Present Senior Research Fellow, Chapin Hall at the University of Chicago 

Honors 

2004 Distinguished Alumni Award - Marywood University 

2005 Distinguished Alumni Achievement Award - Juniata College 

2006 Peter A. Forsythe Award for Leadership in Child Welfare - National Association of Public 

Child Welfare Administrators 

2011 James E. Flynn Prize for Research - University of Southern California School of 

Social Work 

2012 Doctor of Humane Letters, honoris causa, Marywood University 

C. Selected Peer-reviewed Publications 

Wuiczyn, Fred and Britany Orlebeke, Fiscal Reform and Managed Care in Child Welfare: Four Case 
Studies, Policy and Practice, (2000). 

Wuiczyn, Fred, Joseph Kogan, and John Dilts, The Effects of Population Dynamics on Performance 
Measurement, Social Service Review, (2001). 

Wuiczyn, Fred, Kristen Brunner, and Brenda Jones Harden, Infant Placements in Foster Care, Infant 
Mental Health Journal, (2002). 

Baker, Amy, Fred Wuiczyn, and Nan Dale. Covariates of Length of Stay in Residential Treatment. 

Child Welfare. (2003). 

Wuiczyn, Fred. Closing the Gap: Are Changing Exit Patterns Affecting the Time African American 
Children Spend in Foster Care Relative to Caucasian Children. Children and Youth Services 
Review, (2003). 

Wuiczyn, F., Joseph Kogan, and Brenda Jones Harden. (2002). Placement Stability and Movement 
Trajectories. Social Service Review, (2003). 

Wuiczyn, Fred. Reunification from Foster Care. The Future of Children, The David and Lucille 
Packard Foundation: (2004). 

Courtney, Mark, Barbara Needell, and Fred Wuiczyn. Unintended Consequences of the Push for 
Accountability: The Case Of National Child Welfare Performance Standards. Children and Youth 
Services Review. 2004. 

Barth, Richard, Fred Wuiczyn, and Tom Crea. From Anticipation to Evidence: Research on the 
Adoption and Safe Families Act. Virginia Journal of Social Policy and the Law. (2004/2005). 

Wuiczyn, Fred and Emily Zimmerman. Siblings in Foster Care: A Longitudinal Perspective. Children 
and Youth Services Review. 2005. 

Wuiczyn, Fred and Lijun Chen. Adoption Dynamics: Impact of the Adoption and Safe Families Act. 

Social Service Review. (2006). 

Wuiczyn, Fred, Lijun Chen, and Britany Orlebeke. Evaluating Contract Agency Performance. Children 
and Youth Services Review. (2009). 

Wuiczyn, Fred. Epidemiological Perspectives on Child Maltreatment. The Future of Children. 

(2009). 

Wuiczyn, Fred, Cheryl Smithgall, and Lijun Chen. Child Well-being: The Intersection of Schools and 
Child Welfare. Review of Research in Education. (2009). 



Wuiczyn, Fred, Lijun Chen, and Mark Courtney. Family Reunification in Social Structural Context. 

Children and Youth Services Review, 2010. 

Wuiczyn, Fred, Robert Gibbons, Lonnie Snowden, and Bridgette Lery. Poverty Social Disadvantage, 
and Black/White Placement Gap. Children and Youth Services Review, 2012 

D. Research Support 

On-going research support 

Principal Investigator. A Longitudinal Study of Children in Out-of-Home Care. 

A study of developmental trajectories of children placed in out-of-home care. Includes a significant cohort of 
Aboriginal children. 

Principal Investigator, Evaluation of Tennessee Contract Provider Performance. State of Tennessee. 

A study of the performance of individual contract providers with an emphasis on the variability of provider 
performance and the policy and fiscal implications. 

Principal Investigator, The Protect Ohio Title IV-E Project Evaluation, State of Ohio. 

An evaluation of how the state of Ohio implemented the IV-E waiver, with particular emphasis on child safety 
and other outcomes. 

Principal Investigator. Simulation Model of Long Run Demand for Residential Care. Center for State Foster 
Care and Adoption Data (with the Argonne National Laboratory, Center for Advanced Computer Simulation). 
Project supports the development of agent based simulation and the integration of simulation models into the 
FCDA OLAP environment. 

Principal Investigator, Research Support for NYC: Integrating Research, Policy, and Practice. Administration 
for Children’s Services. A project to evaluate New York City's child welfare reform initiatives. 

Completed research 

Principal Investigator. Comparative Analysis of Placement Stability. 

Utilizes the Multi-state Foster Care Data Archive to study placement stability in 14 states. 

Principal Investigator, Evaluation of the Partnership for Social Change - Highbridge. 

A study of foster care placement and maltreatment prevention in a socially disadvantaged neighborhood. 

Principal Investigator, (in collaboration with New York City). An Infant System of Care in Bedford-Stuyvesant. 
U.S. Department of Health and Human Services 

An evaluation of a system of care initiative in one neighborhood in New York City. 

Principal Investigator, The Evidence Base for Child Welfare Policy, The David and Lucille Packard Foundation, 
Funds to support the development of a book manuscript with co-authors Barth, Landsverk, Harden and Yuan. 




Robert Gibbons 

Section of Hospital Medicine 
Professor of Medicine 

Professor of Health Studies and Psychiatry; Director, Center for Health Statistics 


Training 


Degree 

Year 

Institution 

Area 

BA 

1976 

University of Denver 

Chemistry & Mathematics j 

PhD 

1981 

University of Chicago 

Statistics & Psychometrics 


Academic Interests 

Robert Gibbons is a statistician interested in the areas of biostatistics, environmental 
statistics, and psychometrics. Major themes in his work include development of linear 
and non-linear mixed effects regression models for analysis of longitudinal data, analysis 
of environmental monitoring data and inter-laboratory calibration, item response theory 
and computerized adaptive testing, and the development of new statistical methods in 
pharmaco-epidemiology and drug safety. Dr. Gibbons is an elected member of the 
Institute of Medicine of the National Academy of Sciences. 

Representative Publications 


1. Gibbons R.D., & Hedeker D.R. Full-information item bi-factor analysis. 
Psychometrika, 57, 423-436, 1992. 

2. Gibbons R.D., Meltzer D., Duan N. Waiting for Organ Transplantation, Science, 
287, 237-238, 2000. 

3. Gibbons R.D. and Bhaumik D. Weighted random-effects regression models with 
application to inter-laboratory calibration. Technometrics, 43, 192-198, 2001. 

4. Hedeker D. & Gibbons R.D. Longitudinal Data Analysis. New York, John Wiley 
& Sons, 2006. 

5. Gibbons R.D., Segawa E., Karabatsos G., Amatya A.K., Bhaumik D.K., Brown 
C.H, Kapur K., Marcus S., Hur K., Mann J.J. Random-effect Poisson regression 
analysis of adverse event reports: The relationship between antidepressants and 
suicide. Statistics in Medicine, 27, 1814-1833, 2008 














David R. Schwartz, MSW 
Intelligent Outcome Assessment, LLC 



Professional Experience: 

Chief Executive Officer April 2014-Present 

Intelligent Outcome Assessment, LLC 

• Responsible for leading and overseeing an organization focused on developing rigorous 
predictive analytic models and tools for government agencies and corporations. Leads the 
research, development, strategic partnership, and business development functions. 

• Lead predictive analytic model developer for CUNY Hunter College School of Social Work's 
Medicaid outcome prediction and model development project. Currently managing the 
analysis and modeling of 40 million Medicaid beneficiary records in partnership with 
Professor Jonathan Prince at Hunter College School of Social Work. This project was 
funded by the CUNY Hunter College Research Foundation — and one of the project's goals 
is to identify the most salient predictors of inpatient psychiatric hospitalizations for youth 
and the elderly. 

• Responsible for recruiting interdisciplinary research scholars to serve on client model 
development teams. 

• Responsible for cultivating academic research partnerships. 


Director of Research and Evaluation June 2013-March 2014 

William Penn Foundation, Philadelphia, PA 

• Responsible for leading research and evaluation unit at one of the largest foundations in 
the United States with grant centers focused on the environmental protection, K-12/Pre-K 
education, arts funding, arts education, and parks/open spaces. 

• Leader of the Foundation's grantee performance assessment/management research study 
to determine the capacity of the Foundation's grantees to assess their performance and 
leverage data to inform decision-making. The evaluation tool development was informed 
by a series of interviews with foundation performance assessment/management thought 
leaders (e.g.. Center for Effective Philanthropy, Edna McConnell Clark Foundation, Tauck 
Foundation, etc.). 

• Worked closely with the program teams to develop logic models/theories of change to 
serve as a framework for reporting, monitoring, evaluation. The logic models are designed 
to inform the evaluation of individual grants as well as grant portfolios. This requires a 
collaborative process with grantees of mapping/crosswalking their individual logic model 
to the Foundation's portfolio logic models. 

• Successfully launched the Foundation's first online data visualization technology to inform 
staff and board decision-making without external consultants. The data visualization 
technology reads from the Foundation's GIFTS grants management database. 

• Leader of the Foundation's innovative utilization of predictive analytic modeling tools to 
uncover hidden patterns in grantee data. Developed accurate and robust models 
predicting contributed revenue increases, audience growth, and marketing efficiency 
increases using the national Cultural Data Project database. 

• Responsible for managing internal evaluations (e.g., evaluated the planning phase of a 
large watershed protection project), and works with the program teams to make course 
corrections if needed. 

• Responsible for managing external evaluations and vetting external evaluators and their 
proposed methodology. 

• Leader of the Foundation's use of extant data sources to evaluate grant portfolios. 

• Leader and creator of the Foundation's framework for evidence-informed grantmaking. 
This requires coaching and building the capacity of program officers to think critically and 
strategically about the evidence base supporting the interventions prospective grantees 
propose to the Board of Directors. 


• Created new structured reporting and monitoring process to supplant unstructured data 
collection from grantees. The variables draw on logic models that are being developed 
across three grant centers. 

• Responsible for overseeing the evaluation of the Foundation's integrated data system 
investment. 

• Responsible for presenting on the Foundation's research and evaluation strategy. 

• Responsible for developing evidence-based practice toolkit that will be housed on the 
Foundation's website to help grantees learn about where they can find credible research 
syntheses and meta-analyses without being overwhelmed. The toolkit will be developed in 
partnership with research synthesis and translation experts at the Campbell Collaboration. 

• Led the launch of the Foundation's new collaboration tool. 

• Responsible for supervising and building the research and evaluation capacity of junior- 
level analysts. 

• Responsible for meeting with a diverse group of grantees to discuss their proposed 
evaluation designs and logic models. 


Senior Evaluation Consultant January 2012-June 2013 

TCC Group, Philadelphia, PA, New York, NY, San Francisco, CA 

• Responsible for modeling and evaluating the impact of programs. 

• Lead data analytic model developer for nonprofit, foundation, corporate, and government 
clients at TCC Group. 

• Responsible for translating evaluation models for clients, helping them to identify 
pathways to successful and unsuccessful outcomes in their data for program planning and 
decision-making. 

• Data analytics lead for the Gap Foundation's Plan Ahead education project for high school 
students. 

• Arts and Classroom Resources team lead for Target Corporation's Community Relations' 
evaluation and research projects. Developed web-based data collection system that auto¬ 
populates into Excel-based dashboards/scorecards to help the Target Arts team with 
ongoing assessment, planning, and capacity building. Currently conducting research on the 
impact of classroom resources on student outcomes to guide Target's classroom resources 
grant making. 

• Data analytics lead for Tufts Flealth Plan Foundation evaluation project. Developed logic 
model with several grantees and foundation staff to set the stage for data collection. 
Developed universal data collection instrument to evaluate three different program areas 
for the foundation. This project blends quantitative and qualitative research methods. 

• Predictive analytic modeling lead for the State of Florida Department of Children and 
Family Services' Statewide Automated Child Welfare Information System redesign project 
in partnership with IBM. 

• Conducted qualitative research study on nonprofit growth and scaling for Grantmakers for 
Effective Organizations (GEO). This research was recently published by GEO (Pathways To 
Grow Impact: Philanthropy's Role In The Journey). 

• Pioneering the use of data analytics in TCC Group's evaluation and research division. 

• Started up TCC Group's new Data Mining division ($1.2 million in revenue, and the division 
was developed 10 months ago) with Peter York. 

• Experience using data analytics/machine learning to help with survey item reduction. 

• Co-lead on a data analytics project focused on volunteering outcomes for Girl Scouts of 
Northern California. 

• Lead technology and model developer, and project manager, for a national Feeding 
America project. The goal is to develop a dynamic sustainability assessment tool for the 
Feeding America BackPack program. This project blends cutting edge quantitative data 
mining/predictive analytics with qualitative interviews. 

• Developing data analytic models for senior centers across the country, helping center 
administrators and staff to predict negative outcomes as early as possible. 

• Flired by the New Zealand Ministry of Social Development to review their predictive 
analytic modeling work that was based on previous child welfare predictive modeling work 
(Schwartz, Kaufman, Schwartz, 2004). 

• Project manager for the evaluation of Family Reading Partnership's services to the 
community. The first phase of the project is a baseline survey of young parents in the 
Ithaca area. 

President and CEO 2001-2012 

Q-linx, Inc., Government Data Analytics, Philadelphia, PA 
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• Responsible for managing and collaborating with interdisciplinary teams of social science 
researchers, engineers, and programmers working on multiple data analysis and analytics projects in 
Philadelphia, the United States, and internationally. 

• Provided evaluation, data analytics, and recruitment consulting services for two U.S. Department of 
Education Transition to Teaching grants. 

• Worked on a large multi-phase data analytics research project for the New York State Office of 
Children and Family Services (OCFS). Q-linx and IBM were partners on the project. 

• Developed eight predictive models for OCFS based on their CFSR Program Improvement Plan 
outcome, reentry into the child welfare system. The final models were able to predict substantiated 
investigations (and reentry for children with prior investigations) with 91 percent accuracy, and 
unsubstantiated investigations with 90 percent accuracy. 

• Responsible for analyzing OCFS's database and assessment protocols. 

• Presented analytics results to OCFS's Chief Information Officer and other deputy commissioners and 
agency administrators. 

• Flelped to prepare OCFS deputy commissioners for their data analytics research presentation at the 
Yale University Stewards of Change Conference, "From Field to Fed II: Linking Systems to Sustain 
Interoperability in Challenging Times" (3/11), and the New York Public Welfare Association's 
(NYPWA) conference (7/11). 

• Provided expert SACWIS and AFCARS reporting system consulting for Children's Right's D.G. v. Flenry 
class action lawsuit filed on behalf of 10,000 children living in Oklahoma's child welfare system. 
Analyzed database change management processes, database testing/quality assurance, and reports 
(federal and state) generated from the database. 

• Conducted research with Temple University faculty and students from Computer and Information 
Sciences and Criminal Justice departments to develop machine learning-based juvenile delinquency 
risk assessment models utilizing data from the Philadelphia Department of Fluman Services. The 
models reached 90 percent accuracy. The project used integrated data from the School District of 
Philadelphia, the Philadelphia Department of Fluman Services, and Family Court. 

• Working with Temple University computer science professor and Q-linx consultant Dr. Zoran 
Obradovic to develop individual, corporate, and foundation data analytics algorithms to assist 
nonprofits and universities with targeted fundraising efforts. 

• Developing a multidisciplinary, multi-agency operational information exchange and data analytics 
project with Dr. Susan Kinnevy, Deputy Commissioner for the Philadelphia Department of Fluman 
Services, Dr. Don Schwarz, Deputy Mayor for Flealth and Opportunity for the City of Philadelphia, Dr. 
Dennis Culhane, Professor, University of Pennsylvania School of Social Policy and Practice, Dr. 

Richard Gelles, Dean, University of Pennsylvania School of Social Policy and Practice, Dr. Peter Jones, 
Professor and Senior Vice Provost, Temple University Department of Criminal Justice, and Dr. Zoran 
Obradovic, Professor, Computer and Information Sciences Department at Temple University, and 
Director of the Center for Data Analytics and Bioinformatics. 

Founder and Vice President for Research and Development 1998-2001 

Q-linx, Inc., Government Data Analytics, Philadelphia, PA 

• Led and interdisciplinary research project at the University of Pennsylvania exploring the novel 
application of predictive analytic modeling/machine learning techniques to support decision-making 
in human services. Collaborated with a team of professors and students from the University of 
Pennsylvania School of Engineering and Applied Sciences, the Wharton School of Business, and the 
School of Social Policy and Practice. The neural network machine learning technology models 
learned to predict severe child maltreatment with 90 percent accuracy utilizing a nationally 
representative dataset, significantly outperforming existing tools used in the field. 

• Initiated longstanding research, development, and sales partnership between IBM and Q-linx (2000- 
2012) to apply data analytics and machine learning technologies to the prediction and prevention of 
child maltreatment. 
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• Responsible for developing a Wharton Journal award winning business plan for Q-linx, and 
supervising a multi-disciplinary team of students for research assistance. 

• Developed Q-linx risk assessment and online case management prototype. 

• Pilot tested online case management prototype at Volunteers of America Delaware Valley. 

• Conducted research for IBM using the Third National Incidence Study of Child Abuse and Neglect, 
using IBM data analytics technology. 

Director of Development 1997-1998 

Child, Inc., Wilmington, DE 

• Responsible for developing fundraising proposals and evaluation designs for one of Delaware's 
largest nonprofit organizations, serving roughly 20,000 clients per year. 

• Generated over $1 million for a new shelter and support services for victims of domestic violence 
and their children. 

• Conducted research to support funding proposals and agency reports. 

• Provided strategic planning, needs assessment, and program/outcome evaluation services. 

• Developed several successful proposals and presentations to foundations and government agencies. 

Social Work Intern 1996-1997 

Cooper Family Medicine, Camden NJ 

• Helped administrators with the development of an outcome measurement system. 

• Provided case management data analysis services (e.g., analyzing patient visits by disease). 

• Assisted with the selection and development of a management information system. 

Social Work Intern 1995-1996 

Children's Hospital of Philadelphia 

• Case manager in CHOP'S school-based health clinic at University City High School. 

• Provided health prevention and other case management/social work services in the multi¬ 
disciplinary health clinic. 

• Part of a team that ultimately helped University City High School to significantly reduce teen 
pregnancies. 

Case Manager 

The Choice Program, Prince George's County, MD 1994-1995 

• Worked with juvenile delinquents to help them transition into the community. 

• Interfaced with health, probation/parole, school, and child welfare systems. 

• Provided direct services to youth in their home and schools. 

• Worked with at-risk youth in school-based prevention program. Choice Middle Schools. 
Education: 

University of Pennsylvania 
School of Social Policy and Practice 
MSW, Macro Practice Concentration, 1997 

University of Michigan, Ann Arbor 
BA, 1994 

Awards: 

• Wharton Business School Journal, Most Socially Responsible Business Plan Award, Q-linx.com 

( 2000 ). 

• Emerging Practice Research Scholar Award, Third International Conference on Practice 
Research (June, 2014). 
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Research Publications 


• Schwartz, D. R., Kaufman, A. B., & Schwartz, I. M. (2004). Computational intelligence techniques for 
risk assessment and decision support. Children and Youth Services Review, 26,1081-1095. 

• Jones, P. R., Schwartz, D. R., Schwartz, I. M., Obradovic, Z., & Jupin, J. (2006). Risk classification and 
juvenile dispositions: What is the state of the art? Temple Law Review, 79, 2, 461-498. 

• Schwartz, I. M., Jones, P. R., Schwartz, D. R., & Obradovic, Z. Improving Social Work Through the Use 
of Technology and Research, Child Welfare Research, edited by Duncan Lindsey and Aron Shlonsky. 
(2008). Oxford University Press: New York. 

• Schwartz, 1. M., Jones, P. R., & Schwartz, D. R. (2010). Juvenile probation and youth corrections 
trends in the United States and the implications for social work. Korean Journal of Correctional 
Social Welfare, 1,151-169. 

Other Publications 

• Chen, P., Murphy, S., Schwartz, D. R., & Vilhauer, H. (2013). Unlocking the mystery of volunteer 
retention: How girl scouts of Northern California is using advanced predictive analytics to inform its 
volunteer management practices. Stanford Social Innovation Review Blog. 

Selected Presentations 

Numerous presentations and papers on augmenting training, risk assessment, and decision-making with 

technology have been delivered: 


• Using Prediction and Data Mining to Match Clients to Services and Evaluate Programs, Advancing the 
Science of Children's Services through Large Data, First 5 LA/USC/ University of California Berkeley 
Children's Data Network convening. (2014). 

• Blending Quantitative and Qualitative Methods to Predict Volunteer Retention, American Evaluation 
Association. (2013). 

• Uncovering Pathways to Success for Senior Center Participants with Predictive Analytic Modeling, 
American Society on Aging National Conference (with Peter York and Robin Burstein). (2013) 

• Risk Assessment Models and Empirical Validity: Making Life and Death Decisions, Panel Discussion, 
Conference Faculty/Panel Presenter. Qne Child Many Hands Conference, a Multi-Disciplinary Conference 
on Child Welfare, University of Pennsylvania. (2009). 

• Risk Classification in Juvenile Justice and Child Welfare: The Dangers of Qverconfidence. Paper presented 
at the annual meeting of the American Society of Criminology (with Peter R. Jones and Ira M. Schwartz). 
(2008). 

• The International Society for the Prevention of Child Abuse and Neglect, as accepted presenter and invited 
session chair (Berlin, Germany, Warsaw, Poland, and Lisbon, Portugal). 

• Temple University Medical School, Grand Rounds (2008). 

• National Symposium on Child Sexual Abuse, Chair and Presenter (2008). 
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• Facilitated IBM master class presentations, IBM sales team briefings/training, and invited 
presentations at several APHSA Conferences. 

• US Department of Education National Transition to Teaching Conference (2008). 

Competencies: 

IBM SPSS Statistics 19, IBM SPSS Modeler, multi-disciplinary research, CHAID, neural networks, 
classification and regression trees, C5.1, support vector machines, factor analysis/principal 
components analysis, feature selection, predictive analytics, foundation data mining and analytics, 
evaluation project management, nonprofit management, nonprofit fundraising, data analytics and 
evaluation consulting, child welfare and juvenile justice research, advanced data mining, program 
evaluation, blending quantitative and qualitative research, risk assessment, data mapping, 
supervising programmers and researchers, social welfare policy, market research/analysis, 
delivering research presentations, data analytics/machine learning research design, 
fundraising/development data mining, philanthropic social networking technology, proposal 
development/grant writing, strategic planning, business planning, social entrepreneurship, C-level 
meeting facilitation, sales presentations, client relations, IT project management, research and 
development, Microsoft Office, MIS/management information systems, Matlab, MS Project, 
Salesforce 11, Raiser's Edge, SACWIS systems, case management software/systems, case 
management, electronic discovery systems, child protective services (CPS), government consulting, 
business intelligence reporting, Adapa Cloud-based Model Server Engine, Qualtrics, Survey Monkey. 
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E-mail: jpnn@riumeSuny.edu 


Silberman School of Social 
Work at Hunter College 
City University of New York 
2180 3'''Avenue 
New York, NY 10035 


Jonathan D. Prince 


Education 


Ph.D. in Sociai Weifare 

1998 - 2002 The University of Caiifornia, Berkeiey 

■ Speciaiized in mentai heaith services research 

Master’s in Sociai Weifare (M.S.W.) 

1997-1999 The University of Caiifornia, Berkeiey 

■ Management and pianning track 

Master’s in Ciinicai and Community Psychoiogy (M.A.) 

1988 -1991 The University of liiinois, Urbana-Champaign 

■ Speciaiized in psychopathoiogy research 

B.A. in Psychology (honors program) 

1983 -1987 New York University, New York, N.Y. 

■ Minor in computer science and mathematics 

■ Graduated Phi Beta Kappa and Magna Cum Laude 


Professionai Assistant Professor of Sociai Work 

experience Sept., 2009 - Present Hunter Coiiege, City University of New York 

Schooi of Sociai Work 

■ Researching services for peopie with mentai heaith, heaith, or 
substance abuse probiems, risk factors for those probiems, and 
interventions that prevent adverse consequences such as 
hospitaiization or incarceration 

- Examining HIV/AIDS, sexuaiiy transmitted disease, and substance 
abuse in persons with serious mentai iiiness 

■ Researching poiysubstance abuse and outcomes among youth 
(depression, deiinquency, heaith disturbance) and aduits (suicide 
attempt, hospitaiization, physicai iiiness) 

- Anaiyzing 14-state Medicaid data in order to expiore the 
aforementioned topics, and in order to expiore onset of mentai 
iiiness, physicai iiiness, or substance abuse in youth and aduits 
(Principai Investigator: in partnership with Mary McKay at New York 
University) 




■ Evaluating hospitalization and readmission in 10 New York City 
Psychiatric Hospitals (Principal Investigator: in partnership with the 
NYC Department of Health and Mental Hygiene) 

■ Evaluating the post-earthquake mental health, health, and 
substance abuse needs of Haitian residents of New York City 
(Principal Investigator: in partnership with Martha Bragin and the 
NYC Department of Health and Mental Hygiene) 

■ Evaluating two community reentry programs for formerly 
incarcerated adults in New York City (Principal Investigator for one 
program and Co-Pi for the other) 

■ Teaching Human Behavior and the Social Environment I and II, 
Research in Social Work I and II, and a self-developed elective 
entitled Community Mental Healthcare 

■ Chair of the research curriculum in the MSW program 

Assistant Professor of Social Work 

Jan., 2003 - June, 2009 Rutgers: The State University of New Jersey 

School of Social Work 

• Completed and published 16 studies in peer-reviewed journals (e.g., 
American Journal of Public Health, American Journal of Orthopsychiatry, 
Psychiatric Services) that focused on services for people with mood 
disorder, schizophrenia, or other serious mental illness 


■ Taught Human Behavior and the Social Environment, and Research 
Methods in Social Work{\) 

- Developed and taught a course elective entitled Community Mental 
Health Care 

National Institute of Health Pre-doctoral Fellow 

2001 -2002 Center for Self-Help Research, Berkeley, CA, and 

The University of California, Berkeley 

■ Investigated recovery from mental illness in long-term users of Northern 
California self-help agencies 


Doctoral Research Associate 

1998 - 2001 The Bay Area Social Services Consortium, 

University of California, Berkeley 

■ Analyzed Northern California human service agency policy 


Graduate Student Instructor 

1 /2001 -5/2001 The University of California, Berkeley 

■ Taught a discussion section of Introduction to Research 



Acting Director 

1995 - 1997 Conard House, San Francisco, CA 

■ Provided ciinicai and administrative supervision to community mentai 
health practitioners in the Director’s absence 


Coordinator of Ciient Services 

1995 -1997 Conard House, San Francisco, CA 

■ Compieted intake interviews and oversaw activities of direct service staff 
in the deiivery of ciient services 

Counseior/Senior Counseior 

1992 -1995 Conard House, San Francisco, CA 

- Provided individuai and group counseiing to aduits with psychopathoiogy 


Course Instructor 

1990 - 1991 The University of liiinois, Urbana-Champaign 

■ Taug ht Abnormal Psychology 


Pubiications 

Journal Articles -In press, under review, In process, or published (In 
reverse chronological order) 

Prince, J.D. (2013). Caii for research: detecting eariy vuinerabiiity for 
psychiatric hospitaiization. Journal of Behavioral Health Sen/Ices and 
Research, 40, 46-56. 

Prince, J.D. (2012). Risk of inpatient stay for mentai iiiness among 
individuais with substance use disorders. Psychiatric Services, 63, 938- 
941. 

Prince, J.D., Akincigii, A., Waikup, J.T., Amin, S., Crystai, S. (2012). 
Serious mentai iiiness and risk of new HIV/AIDS diagnoses: An anaiysis 
of Medicaid beneficiaries in eight states. Psychiatric Services, 63, 1032- 
1039. 

Prince, J.D., Akincigii, A., Waikup, J.T., Amin, S., Crystai, S. (in press). 
Sexuaiiy transmitted disease among persons with HIV/AIDS in eight 
states: Is severe mental illness a risk factor? Psychiatric Services 

Prince, J.D. (in press). Opioid analgesic use disorders among 
adolescents in the United States. Journal of Child and Adolescent 
Substance Abuse. 

Prince, J.D. & Lalayants, M. (in press). Delinquency, depression, and 
substance abuse among child welfare-involved adolescent females. Child 
Abuse and Neglect 


Prince, J.D. (under review). Misuse of opioid analgesics versus other 
drugs in the United States. 



Lalayants, M. & Prince, J.D. (under review). Loneiiness and depression or 
depression-reiated factors among chiid weifare-invoived adoiescent femaies. 

Prince, J.D., Akincigii, A., Hoover, D.R., Waikup, J.T., Biider, S., Crystai, S. 
(2009). Substance abuse and hospitaiization for mood disorder among 
Medicaid beneficiaries. American Journal of Public Health, 99,160-167. 

Prince, J.D., Akincigii, A., Hoover, D.R., Lucas, J., Bowbiis, J., Waikup, J.T., 
Kaiay, E., Crystai, S. (2008). Psychiatric re-hospitaiization among eideriy 
persons in the United States. Psychiatric Sen/ices, 59, 1038-1045. 

Akincigii, A., Hoover, D.R., Waikup, J.T., Prince, J.D., Kaiay, E., Crystai, S. 
(2008). Hospitaiization for psychiatric iiiness among community dweiiing 
persons in 1992-2002. Psychiatric Sen/ices, 59, 1046-1048. 

Hoover, D.R., Akincigii, A., Prince, J.D., Kaiay, E., Lucas, J., Waikup, J.T., 
Cystai, S. (2008). Medicare inpatient treatment for eideriy non-dementia 
psychiatric iiinesses 1992-2002: Length of stay and expenditures by faciiity 
type. Administration and Policy in Mental Health, 35, 231 -240. 

Prince, J.D., Akincigii A., Bromet E. (2007). Incarceration rates of persons 
with first-admission psychosis. Psychiatric Sen/ices, 58, 1173-1180. 

Prince, J.D. (2007). Education, ethnicity, and subjective iife quaiity of recentiy 
discharged inpatients with schizophrenia. Journal of Nervous and Mental 
Disease, 195, 560-575. 

Prince, J.D. (2007). Therapeutic aiiiance, iiiness awareness, and number of 
hospitaiizations for schizophrenia. Journal of Nervous and Mental Disease, 
195, 170-174. 

Prince, J.D. (2007). Promoting consumer empowerment through 
entrepreneurship: A proposai. Psychiatric Rehabilitation Journal, 30, 223- 
225. 

Prince, J.D. (2006). Ethnicity and iife quaiity of recentiy discharged inpatients 
with schizophrenia. American Journal of Orthopsychiatry, 76, 202-205. 

Prince, J.D. (2006). Incarceration and hospitai care. Journal of Nen/ous and 
Mental Disease, 194,34-39. 

Prince, J.D. (2006). Determinants of care satisfaction among inpatients with 
schizophrenia. Community Mental Health Journal, 42, 189-196. 

Prince, J.D. (2006). Practices preventing re-hospitaiization of inviduais with 
schizophrenia. Journal of Nervous and Mental Disease, 194, 397-404. 

Prince, J.D. (2005). Predicting outpatient mentai heaith program withdrawai 
among recentiy discharged inpatients with schizophrenia. Psychological 
Services, 2, 142-150. 

Prince, J.D. (2005). Life contentment and mentai heaith care satisfaction. 
Research on Social Work Practice, 15, 564-567. 



Prince, J.D. (2005). Family involvement and satisfaction with community 
mental health care of individuals with schizophrenia. Community Mental 
Health Journal, 41, 419-430. 

Prince, J.D. & Austin, M.J. (2005). Inter-agency collaboration in child welfare 
and child mental health systems. Social Work in Mental Health, 4, 1 -19. 

Baer, J., Prince, J.D. & Valez, J. (2004). Fusion or familialism: A construct 
problem in studies of Mexican-American adolescents. Hispanic Journal of 
Behavior Sciences, 26, 263-273. 

Austin, M.J. & Prince, J.D. (2003). The implications of managed care and 
welfare reform for the integration of health and welfare services. Journal of 
Health and Social Policy, 18, 1-19. 

Prince, J.D. & Austin, M.J. (2001). Innovative programs and practices 
emerging from the implementation of welfare reform: A cross-case analysis. 
Journal of Community Practice, 9, 1-14. 

Brief Commentary 

Prince, J.D. (2013). Critical time intervention reduces psychiatric re¬ 
hospitalization among formerly homeless individuals with psychotic 
diagnoses (commentary). Evidence-Based Mental Health, 16, 38. 

Prince, J.D. (2012). HIV/AIDS and serious mental illness: A risky conclusion: 
In reply (letter). Psychiatric Sen/ices, 63,1261. 

Prince, J.D. (2006). Policy challenges in modern health care (book review). 
Journal of Nervous and Mental Disease, 194, 721 -722. 

Prince, J.D. (2005). Efforts to encourage entrepreneurship among persons 
with mental illness. Psychiatric Sen/ices, 56, 359-360. 

Prince, J.D. & Segal, S.P. (2005). An empowering medical model of mental 
health care. Psychiatric Sen/ices, 58, 901. 

Book Chapters 

Prince, J.D. & Austin, M.J. (2004). Overview of innovative programs and 
practices. In M.J. Austin, ed. Changing Welfare Sen/ices: Case Studies of 
Local Welfare Reform Programs. New York: The Haworth Press, Chapter 2. 

Prince, J.D. & Austin, M.J. (2004). Training exempt providers to deliver high- 
quality child care programs. In M.J. Austin, ed. Changing Welfare Sen/ices: 
Case Studies of Local Welfare Reform Programs. New York: The Haworth 
Press, Chapter 5. 

Prince, J.D. & Austin, M.J. (2004). Combining business with rehabilitation in 
a public work center for disabled and low-income participants. In M.J. Austin, 
ed. Changing Welfare Sen/ices: Case Studies of Local Welfare Reform 
Programs. New York: The Haworth Press, Chapter 7. 



Prince, J.D. & Austin, M.J. (2004). Merging a workforce investment board 
and a department of sociai services into a county department of empioyment 
and human services. In M.J. Austin, ed. Changing Welfare Sen/ices: Case 
Studies of Local Welfare Reform Programs. New York: The Haworth Press, 
Chapter 21. 

Prince, J.D. & Austin, M.J. (2004). Crossover services between chiid weifare 
and weifare-to-work programs. In M.J. Austin, ed. Changing Welfare 
Sen/ices: Case Studies of Local Welfare Reform Programs. New York: The 
Haworth Press, Chapter 23. 

Grants 

2011-Present: Principai Investigator (with two Co-PIs from the New York City 
Department of Heaith and Mentai Hygiene) on a grant evaiuating the 
effectiveness of New York City psychiatric hospitais ($545,000) 


2011-2012: Principai Investigator (with Martha Bragin, Ph.D.) on a federai 
Emergency Response Grant from the Substance Abuse and Mentai Heaith 
Services Administration (in partnership with the New York City Department of 
Heaith and Mentai Hygiene) evaiuating outreach and referrai services for 
Haitian immigrants iiving in New York City with mentai heaith or substance 
abuse probiems reiating to the Haitian earthquake ($318,000) 

2011-Present: Principai investigator for research assessing psychiatric 
service use and outcomes among Medicaid beneficiaries in 14 American 
states (with Mary McKay, Ph.D. at New York University ($32,000) 


2013-Present: Principai Investigator - Presidentiai Fund for Facuity 
Deveiopment Award at the City University of New York: Researching 
questionabie prescription practices among Medicaid beneficiaries in 14 
American States ($3000) 

2011-Present: Principai Investigator on an evaiuation of a prisoner reentry 
program in New York City (Greenhope Services for Women) ($25,000 per 
year) 

2011-Present: Co-investigator (with Anthony Dejesus, Ph.D. and George 
Patterson, Ph.D.) on an evaiuation of SAMHSHA services for addressing 
chronic homeiessness and HIV among formerly incarcerated women in the 
Greenhope Housing Development Association ($50,000 per year) 


2010-2011: Principal Investigator on a grant from the Professional Staff 
Congress-City University of New York Research Award Program to 
study psychotherapy use and the prevention of psychiatric 
hospitalization ($6000) 



2009-2010: Principal Investigator on a grant from the Professional Staff 
Congress-City University of New York Research Award Program to 
study substance abuse and psychiatric hospitalization ($4000) 

2003-2010: Received a two-year grant, and two two-year renewals (totaling 
about $90,000), from the National Institute of Health’s Loan Repayment 
Program (LRP) 

Conference Presentations 

Prince, J.D., & Lalayants, M. Delinquency, Depression, and Substance Use 
Disorder among Child Welfare-Involved Adolescent Females. Council on 
Social Work Education, Dallas, Texas, 10/31-11/3. 

Lalayants, M. & Prince, J.D. Loneliness and Depression or Depression- 
Related Factors among Child Welfare-Involved Adolescent Females. Council 
on Social Work Education, Dallas, Texas, 10/31-11/3. 

Patterson, G., Dejesus, A. & Prince, J.D. Use of the Level of Service 
Inventory-Revised (LSI-R) among Female Offenders. Council on Social 
Work Education, Dallas, Texas, 10/31-11/3. 

Prince, J.D., Walkup, J.T., Amin, S., Elkowitz, J., Crystal, S. (2009). Risk for 
HIV infection among Medicaid Beneficiaries with Serious Mental Illness. 
Society for Social Work Research, Washington, D.C., 1/11/12-1/15/12. 

Prince, J.D., Walkup, J.T., Amin, S., Elkowitz, J., Crystal, S. (2009). 
Predicting HIV/AIDS: Is Serious Mental Illness a Risk Factor in Medicaid 
Beneficiaries? American Public Health Association Annual Conference, 
Boston, MA., 11/4/09-11/8/09 

Prince, J.D. (2007). Incarceration of persons with first admission psychosis. 
The Eighth Workshop on Costs and Assessments in Psychiatry: Investing in 
Mental Health Policy and Economics Research, Venice, Italy, 3/09/07- 
3/11/07 

Prince, J.D. (2006). Predicting outpatient program withdrawal among 

recently discharged individuals with schizophrenia. American Public Health 
Association Annual Conference, Boston, MA, 11/4/06-11/8/06 

Prince, J.D. (2006). Predicting outpatient program withdrawal among 

recently discharged individuals with schizophrenia. Society for Social Work 
Research, San Antonio, Texas, 1/12/06 -1/15/06 

Prince, J.D. (2006). Ethnicity and life quality among recently discharged 
inpatients with Schizophrenia. Annual Program Meeting of the Council on 
Social Work Education, Chicago, Illinois, 2/16/06-2/19/06 

Prince, J.D. (2002). Innovations in welfare reform implementation: A cross¬ 
case analysis. Annual Program Meeting of the Council on Social Work 
Education, Nashville, Tennessee, 2/24/02-2/27/02 



Peer-Reviewer for Manuscripts Submitted to Journais 

American Journal of Public Health 
Journal of Public Health 

Bulletin of the World Health Services Organization 

Archives of General Psychiatry 

The American Journal of Orthopsychiatry 

Psychiatric Services 

Drug and Alcohol Dependence 

BMC Psychiatry 

The Psychiatric Rehabilitation Journal 

Psychological Services 

The Journal of Nervous and Mental Disease 

The Community Mental Health Journal 

Research on Social Work Practice 

Journal of Behavioral Health Services and Research 

Evidence-Based Mental Health 

Journal of Teaching in Social Work 



Tool Descriptions 

Overview of Computerized Adaptive Mental Health Testing 
lOMA - Intelligent Outeome Management and Assessment 
IBM SPSS Modeler Professional 

IBM SPSS Modeler Server performanee and optimization 

Agent-Based Soeial Simulation in the Serviee of Child Welfare Issues: Defining the Demand for 
Group and Residential Care 


Deeision Support Tools and Predietive Analyties in Human Serviees Kempe Center Bid 
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Overview of Computerized Adaptive Mentai Heaith Testing 

The Computerized Adaptive Test-Mental Health or CAT-MH, is currently a collection of three 
adaptive tests for depression, anxiety, and mania, and a diagnostic screening test for major 
depressive disorder (CAD-MDD) developed as part of a 5-year grant from the National Institute 
of Mental Health. The CAD-MDD produces a screening diagnosis of depression and a 
corresponding confidence level associated with that diagnosis. By contrast, the three 
computerized adaptive tests (CAT), the CAT-Depression Inventory or CAT-DI, the CAT-Anxiety 
or CAT-ANX and the CAT-bipolar disorder or CAT-BP, are dimensional measures that produce 
continuous severity scores based on symptomatology experienced in the past two weeks. The 
paradigm shift between traditional screening and assessment tools and those associated with 
these tests is that they begin with a large “bank” of items (1008 psychiatric symptom items) and 
adaptively administer a small and statistically optimal subset of the items (on average 12 items 
for each of the three CATs and 4 items for the CAD-MDD). Nevertheless, each of the CATs 
maintains a correlation of close to r=0.95 with the entire bank of items for each test (389 
depression items, 431 anxiety items, 88 bipolar items). As such, with only 12 items in 2 minutes 
over the internet, we can extract the information contained in hundreds of items in the item 
bank. Results for the CAT-DI (depression test) have recently been published in JAMA 
Psychiatry, the CAT-ANX in the American Journai of Psychiatry, and the CAD-MDD in the 
Journai of Ciinicai Psychiatry. In terms of our ability to screen for depression, the CAD-MDD 
has remarkably high sensitivity of 0.95 and specificity of 0.87 with an average of only 4 items 
(max=6 items). As a comparison, the PHQ-9 used in these same subjects had similar 
specificity, but sensitivity of 0.70 indicating that for every 100 true cases the PHQ-9 would miss 
30, whereas the CAD-MDD would only miss 5. This advantage is achieved despite the fact that 
the CAD-MDD uses less than half the number of items. Patients taking any of these tests are 
also screened for suicide risk using the Columbia Suicide Severity Rating Scale (C-SSRS) 
screener. 

The ultimate goal of the CAT-MH is to make diagnostic screening of mental disorders 
such as depression and long term monitoring of the severity of those disorders as seamless as 
possible, decreasing patient and clinician burden to near zero levels. The CAT-MH will be 
distributed as a cloud-computing system where a secure and HIPAA compliant internet system 
is used to assess individuals anytime from anywhere on any internet-capable device. For 
example, the cloud will permit an individual in urban Chicago, the most remote corner of the 
Navajo Nation or rural West Virginia to log on 24x7x365 via an internet browser with a mobile 
computing device, a laptop or a desktop to a secure website with a unique ID and password and 
take a test. The results of that test and automated suicide warnings can be directly transmitted 
to any number of reviewers in near real time and can be placed directly into the patient’s 
electronic medical record (EMR). Weekly monitoring of depressive severity can be performed 
following a prescription of an antidepressant to insure that the treatment is functioning as 
intended and this information can be transmitted automatically to the treating physician. It 
should be obvious that the implementation of the CAT-MH is almost as important as its scientific 
foundation. A major advantage of CAT in this regard is that repeat testing need not involve the 
same items, and the starting value for the next session can be informed by the score on the 
previous session leading to further reduction in patient burden. 

1. Gibbons R.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kim J.B., Kupfer D.K. Development 
of a computerized adaptive test for depression. JAMA Psychiatry, 2012;69,1104-1112. 

2. Gibbons R.D., Hooker G., Finkelman M.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kupfer .J. The CAD-MDD: A 
computerized adaptive diagnostic screening tool for depression. Journal of Clinical Psychiatry, 74, 669-674, 2013. 
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Case Study Spectrum 


Mental 

Education Health 



Social 

Services 



Juvenile 

Justice 



Adult 

Justice 



Government and Education 


Community Care/Program Providers 



Drop-out 

• Hospitalization 

• Cost-effective 

• Detention 

• Adult Recidivism 

Prevention 

• Costly 

Placements 

Alternatives 

• Cost-effective 

School 

Community- 

• Child Protective 

• Disproportionate 

Probation and 

Violence 

based 

Services 

Minority Contact 

Parole 

Prevention 

Placements 

• Cross-over into 

• Juvenile 

Placements 

School-level 

• Inappropriate 

Justice 

Probation 

• Drug (Courts) 

Performance 

Medication 



Administration 


Assessment •Transitioning 

Youth • Gang 

Identification and 
Monitoring 
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Intelligent Outcome Management and Assessment (lOMA) 


Intake 



Evidence-Based 
Predictive Modeiing 



Outcome Pathways 



Smart Intervention • Outcome Prediction • Forecasting • Consumer Insights 


• Enables outcome 
determination at point-of- 
service 


• Utilizes native variables and factors; 
not a one-size-fits-all-jurisdictional 
model 


• Provide frontline 
practitioner discernible 
action plan 


• Utilizes smart intake 
scoring algorithms that 
learn as intake process 
unfolds. 

• Leverages existing 
intervention forms or 
case management 
systems 


• Dynamic machine learning system 

• Augments existing risk assessment 
modeling (i.e. SDM) 

• Supports program/provider analysis to 
achieve next-generation risk 
assessment modeling (combining client 
risk with dynamic provider performance 
data) 


• Intelligent, customizable 
dashboard console for 
operational point-and- 
click performance 
monitoring 

• Fully integrated with 
rules-based optimization 
engines. 
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IBM SPSS Modeler 
Professional 

Make better decisions through predictive intelligence 


Highlights 

Create more effective strategies by 
evaluating trends and likely outcomes. 

• Easily access, prepare and model 
structured data with this intuitive, 
visual data mining workbench. 

• Rapidly build and validate models using 
the most advanced statistical and 
machine-learning techniques available. 

• Efficiently deploy insight and predictive 
models. 

• Seamlessly deploy results directly into 
IBM® Cognos® Business Intelligence. 


Data mining provides organizations with a clearer view of current 
conditions and deeper insight into future events by analyzing historical 
data. IBM SPSS® Modeler Professional is a data mining workbench 
used to analyze structured data to create predictive intelligence. This 
intelligence allows business decisions to be made based on the data, 
rather than “gut feel”. 

Businesses of all types have found that they can use the predictive 
intelligence of IBM SPSS Modeler to attract customers, strengthen 
their loyalty, reduce customer attrition or “churn” more cost effectively, 
decrease fraud and minimize risk. Public sector organizations have 
benefited from using Modeler to predict workforce capacity, proactively 
respond to public safety issues, manage the student lifecycle, improve 
classroom performance and address many other operational challenges. 


• Present analytic results across regions 
visually with geographical mapping. 

• Perform in-database analytics with 
leading databases. 


Predictive intelligence enables you to look into the future. Using 
historical data, SPSS Modeler lets you confidently predict outcomes 
and gain an imderstanding about the relationships hidden in your data. 
This understanding allows you to solve any business problem faster 
using powerful, proven analytical techniques that deliver deeper insight 
into your customers, students or constituents. 
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Streamline the data mining process 

SPSS Modeler Professional is popular with analysts and 
business users alike. Its automated data preparation and 
modeling features enable non-analysts to produce accurate 
models quickly and easily without specialized skills. While 
professional analysts can take advantage of the software’s 
advanced data preparation and predictive modeling 
capabilities to create the most sophisticated of streams. 

Organization benefits 

Using SPSS Modeler Professional, all types of organizations 
are performing more in-depth analyses that help them gain 
insight to achieve greater success. 

• Businesses attract customers, strengthen their loyalty, 
reduce customer attrition more cost effectively, identify 
and prevent fraud and reduce risk. 

• Public sector organizations predict workforce capacity, 
evaluate program effectiveness and proactively respond 
to public safety issues. 


• Educational instimtions manage the student lifecycle, 
improve classroom performance and address many 
other student and operational challenges. 

• Industrial operations refine maintenance planning to 
prevent unscheduled downtime. 

The intuitive graphical interface of SPSS Modeler makes 
it easy for users to visualize every step of the data mining 
process as part of a “stream.” By interacting with these 
streams, analysts and business users can collaborate-adding 
business knowledge and domain expertise to the data mining 
process. It allows them to focus on discovering insights 
rather than on technical tasks like writing code. They can 
also pursue “train-of-thought” analysis, explore the data 
more deeply; uncovering additional relationships that make 
sense to the organization. 
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Figure 1: The SPSS Modeler visual interface enabies users to build predictive modeis quickiy and intuitiveiy, without the need for programming. 
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Easily access and integrate data 

From this visual interface, you can easily access and integrate 
data in virtually any type of database, spreadsheet or flat file, 
including IBM SPSS Statistics, IBM SPSS Data Collection, 
Cognos Business Intelligence, SAS and Microsoft Excel flies. 

When combined vtith SPSS Modeler Professional Server, 
there is no need to move data from large databases, since 
the analytics and mining take place in-database. The result 
is a significant improvement in analytical performance. 

SQL Pushback allovts for data transformation and 
preparation tasks to be performed vtithin the database 
vtithout the user having to vtrite any SQL or do any 
programming. Additionally, algorithms from leading 
databases can be accessed directly w^ith the user-friendly 
SPSS Modeler interface and built and scored as part of a 
Modeler stream-supported vtith IBM InfoSphere®, Microsoft 
SQL Server, Qracle and IBM Netezza. SPSS Scoring 
Adapters allovt the data to be scored in-database, resulting 
in quicker decisions and better RQI. 

Integration with IBM Cognos software 

Analysts can access data from their Cognos Business 
Intelligence environment directly w^ithin the SPSS Modeler 
interface. IBM Cognos software organizes and delivers 
a complete and consistent view of information for 
enterprise-wide decision making. By adding the analytic 
capabilities of SPSS Modeler Professional, organizations 
can quickly and reliably evaluate the likelihood of specific 
outcomes, using their familiar enterprise-wide data view. 


Also, since SPSS Modeler can write results into Cognos 
Business Intelligence-they can make predictive intelligence 
available to business users and all information stakeholders 
who rely on Cognos as their information portal into 
enterprise analytics. 

Choose from an unparalleled breadth 
of techniques 

SPSS Modeler offers an array of advanced data mining 
techniques that are designed to meet the needs of every data 
mining application, including all of the following algorithms. 

• Classification algorithms. Make predictions or forecasts 
based on historical data using techniques such as Decision 
Trees, Neural Networks, Logistic Regression, Support 
Vector Machines, Cox regression. Generalized Linear Alixed 
Models (GLMM) and more. Use automatic classification 
modeling for both binary and numeric outcomes to 
streamline model creation. 

• Segmentation algorithms. Group people or detect unusual 
patterns with automatic clustering, anomaly detection and 
clustering neural network techniques. Use automatic 
classification to apply multiple algorithms with a single step 
and take the guesswork out of selecting the right technique. 

• Association algorithms. Discover associations, links or 
sequences using Apriori, CARMA and sequential association. 

• Time series and forecasting. Generate forecasts for one or 
more series over time using statistical modeling techniques. 
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Optimize your current information 
technologies 

With its open and scalable architecture, SPSS Modeler 
makes the best use of your existing IT infrastructure. 

It integrates with your existing systems, both when accessing 
data and when deploying results, so you don’t need to 
move data into and out of a proprietary format. Additionally, 
techniques such as in-database mining, SQL pushback, 
multi-threading, server clustering and in-database scoring 
help conserve resources, deliver results faster and reduce 
overall IT costs. 

Follow a proven, repeatable process 

During every phase of the data mining process, SPSS Modeler 
supports the de facto industry standard, the CRoss-Industry 
Standard Process for Data Mining (CRISP-DM). This means 
users can focus on solving business problems through data 
mining, rather than on reinventing a new process for every 
project. Individual Modeler projects can be efficiently 
organized using the CRISP-DM project manager. 



Figure 2: The CRISP-DM process, as shown in this diagram, enables data 
miners to implement efficient data mining projects that yield measurable 
business results. 
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Deploy predictive modeling across 
the enterprise 

SPSS Modeler can efficiently analyze the amounts of data 
typically generated by small to mid-sized organizations. 
Organizations with high-volume or complex data mining 
requirements can take advantage of the additional power 
of SPSS Modeler Server. Using client/server architecture, 
SPSS Modeler Server allows many data analysts to work 


simultaneously without straining computing resources. You 
can take advantage of in-database mining and in-database 
scoring on leading platforms and efficiently process large 
amounts of data. SPSS Modeler Server also offers additional 
deployment options to help you extend the benefits of 
data mining across geographic or functional lines and put 
results in the hands of decision makers quickly. 


IBM SPSS Modeler Professional features 

Data 

understanding 

• Create a wide range of interactive graphs with automatic assistance 

• Use visuai iink anaiysis to see associations in your data 

• Interact with data by seiecting regions or items on a graph and viewing the seiected information; or seiect key data for use in anaiysis 

• Access SPSS Statistics graphs and reporting toois directiy from the SPSS Modeier interface 

Data 

preparation 

• Access operational data from Cognos Business Intelligence, IBM DB2® Oracle, Microsoft SQL Server, IBM Informix®, IBM Netezza, 
mySQL (Oracle) and Teradata data sources, as well as mainframe data through zDB2 and IBM Classic Federation Server support 

• Import delimited and fixed-width text files, SPSS Statistics files, SPSS Data Collection data sources. Excel, SAS or XML 

• Choose from the multiple data-cleaning options available in SPSS Modeler to remove or replace invalid data, automatically impute 
missing values and mitigate outliers and extremes 

• Apply automatic data preparation to interrogate and condition data for analysis in a single step 

• Access data management and transformations performed in SPSS Statistics directly from SPSS Modeler 

• Use field filtering, naming, derivation, binning, re-categorization, value replacement and field reordering 

• Apply record selection, sampling (including clustered and stratified sampling), merging (including inner joins, full outer joins, partial 
outer joins, and anti-joins), sorting, aggregation and balancing 

• Choose from options for data restructuring, partitioning and transposition 

• Select from extensive string functions: string creation, substitution, search and matching, whitespace removal and truncation 

• Apply RFM (Recency, Frequency, and Monetary) scoring: aggregate customer transactions to provide RFM value scores and 
combine these to produce a complete RFM analysis 

• Export data to databases, IBM Cognos Business Intelligence packages, SPSS Statistics, SPSS Data Collection, delimited text files. 

Excel, SAS, or XML 
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IBM SPSS Modeler Professional features (continued) 


Modeling 

algorithms 

included 

• Anomaly Detection - Detect unusual records through the use of a cluster-based algorithm 

• Apriori - Popular association discovery algorithm with advanced evaluation functions 

• Bayesian Networks - Graphical probabilistic models 

• C&RT, C5.0, CHAID and QUEST - Decision tree algorithms including interactive tree building 

• CARMA - Association algorithm which supports multiple consequents 

• Cox regression - Calculate likely time to an event 

• Decision List - Interactive rule-building algorithm 

• Factor/PCA, Feature Selection - Data reduction algorithms 

• In-database mining algorithms for IBM InfoSphere*: Association, Clustering, Decision Tree, Logistic Regression, Naive Bayes, 
Regression, Sequence, Time Series 

• In-database mining algorithms for IBM Netezza*: Bayes Net, Decision Trees, Divisive Clustering, Generalized Linear, K-Means, 

KNN, Linear Regression, Naive Bayes, PCA, Regression Tree, Time Series 

• In-database mining algorithms for Microsoft SQL Server*: Association Rules, Clustering, Decision Tree, Linear Regression, 

Naive Bayes, Neural Network, Sequence Clustering, Time-Series 

• In-database mining algorithms for Oracle*: Adaptive Bayes, Apriori, Artificial Intelligence (Al), Decision Tree, General Linear Model 
(GLM), KMeans, Minimum Description Length (MDL), Naive Bayes, Non-Negative Matrix Factorization, 0-Cluster (Orthogonal 
Partitioning Clustering), Support Vector Machine 

• K-Means, Kohonen, Two Step, Discriminant, Support Vector Machine (SVM) - Clustering and segmentation algorithms 

• KNN - Nearest neighbor modeling and scoring algorithm 

• Logistic Regression - For binary outcomes 

• Neural Networks - Multi-layer perceptrons with back-propagation learning, and radial basis function networks 

• Regression, Linear, GenLin (GLM), Generalized Linear Mixed Models (GLMM) - Linear equation modeling 

• Self-learning response model (SLRM) - Bayesian model with incremental learning 

• Sequence - Sequential association aigorithm for order-sensitive analyses 

• Support Vector Machine - Advanced algorithm with accurate performance for wide datasets 

• Time-series - Generate and automatically select time-series forecasting models 

Modeling and 

evaluation 

• Employ a wide range of data mining algorithms with many advanced features to get the best possible results from your data. 

• Use automatic classification (binary and numeric) and clustering in place of selecting individual algorithms 

• Use interactive model and equation browsers and view advanced statistical output 

• Show relative impact of data attributes on predicted outcomes with variable importance graphs 

• Visualize the analytic results on geographic maps 

• Combine multiple models (ensemble modeling) or use one model to analyze a second model 

• Use the SPSS Modeler Component-Level Extension Framework (CLEF) to integrate custom algorithms 

• Use the R statistical programming language to extend analysis options, through the integration of SPSS Statistics 

Deployment 

• Export models using SQL or PMML (the XML-based standard format for predictive models) 

• Use IBM SPSS Collaboration and Deployment Services for innovative analytics management, process automation and 
deployment capabilities 

Modeler 

server* 

• Use in-database mining to build models in the database using leading database technologies and leverage high-performance 
database implementations 

• Use SQL-pushback to push data transformations and select modeling aigorithms directly into your operational databases 

• Leverage high-performance hardware, including IBM System z® machines, to experience quicker time-to-solution and achieve 
greater ROI through parallel execution of streams and multiple models 

• Transmit sensitive data securely between SPSS Modeler Client and SPSS Modeler Server through secure socket layer (SSL) 
encryption 


"Requires Modeler Professional Server 
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About IBM Business Analytics 

IBM Business Analytics software delivers actionable insights 
decision-makers need to achieve better business performance. 
IBM offers a comprehensive, unified portfolio of business 
intelligence, predictive and advanced analytics, financial 
performance and strategy management, governance, risk 
and compliance and analytic applications. 

With IBM software, companies can spot trends, patterns and 
anomalies, compare “what if’ scenarios, predict potential 
threats and opportunities, identify and manage key business 
risks and plan, budget and forecast resources. With these deep 
analytic capabilities our customers around the world can 
better understand, anticipate and shape business outcomes. 

For more information 

For further information please visit ibm.com/business- 
analytics. 

Request a call 

To request a call or to ask a question, go to ibm.com/ 
business-analytics/contactus. An IBM representative 
will respond to your inquiry within two business days. 
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Introduction 

Data mining offers organizations many benefits, including a 
more detailed view of their customers or assets, a clearer view 
of current conditions and deeper insight into future events. By 
choosing a high-performance data mining tool, organizations 
can mine their data more efficiently and gain a significant 
return on investment (ROI) by understanding the patterns in 
their data, allowing better decisions to be made which drive 
superior outcomes. 

IBM® SPSS® Modeler is a data mining workbench that 
enables improved decision making with quick development 
of predictive models using business expertise and quick 
deployment of these models into business operations. 
Designed around the industry-standard CRISP-DM (CRoss 
Industry Standard Process for Data Mining) model, SPSS 
Modeler supports the entire data mining process, from 
business understanding through to deployment. 

SPSS Modeler uses a client/server architecture, allowing 
resource-intensive operations to be executed on a server. 

This results in increased performance because memory¬ 
intensive operations can be done on the server without 
downloading data to the client computer. IBM® SPSS® 
Modeler Server also provides support for SQL optimization 
and in-database modeling capabilities, delivering further 
benefits in performance.’ 


23 Conclusion 

23 About IBM Business Analytics 
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This benchmarking document provides a guide to SPSS 
Modeler Server’s capabilities and possibilities, and serves as a 
guide to understanding and maximizing SPSS Modeler Server 
performance. Thus, the initial sections of this document focus 
on SPSS Modeler Server’s out-of-the-box performance while 
the remaining sections describe more general SPSS Modeler 
performance optimizations and sizing requirements. 

Specifically, many of the results provided in this document 
address SPSS Modeler Server performance as it relates to 
issues of scalability. By utilizing options only available in the 
server version (such as SQL pushback/generation, in-database 
mining and caching. User Defined Functions (UDFs) and very 
large dataset (VLD) options), SPSS Modeler Server-enabled 
streams demonstrate improved performance and a quicker 
return on IT investment. 

Performance and scalability 

SPSS Modeler Server has been designed and developed to 
provide high performance and scalability for all data mining 
tasks. For example, SQL generation and parallel processing are 
performed automatically. As a result, SPSS Modeler users do 
not need to make any changes to the way they work to get 
consistently high performance. 


Td benchmark performance, we measured the ability of 
SPSS Modeler Server to carry out the common tasks of data 
preparation, model building and model scoring using a variety 
of operating environments and altered the size of the data 
files.^ Data mining involves more than simply model building 
and model scoring, with a large component of the process 
involving data preparation. Therefore this paper also includes 
tests evaluating the performance of common steps such as 
reading, sorting, aggregating, merging and writing data. 

Reading data 

Times have been recorded for reading the data sets into SPSS 
Modeler using the stream shown in Figure 1. The Sample 
node in these test streams means that we are able to record 
just the time taken to read the data in (no data write time 
was added). 



TextFile SampleNoRecords DataOut 


Figure 1 : Test stream to read the data into SPSS Modeler. 
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Many of our benchmark tests show that SPSS Modeler Server’s 
performance is consistent when different types of datasets 
(such as CSV files and database tables) are used and when these 
sets are scaled by the number of rows or columns (Figure 2). 
The execution times show that SPSS Modeler performs in a 
relatively linear manner as the data sets are scaled up for CSV 
files, SPSS .sav and database tables. 


The performance of Excel and XML file formats were also 
tested, however, because the reading performance was not 
expected to be as good as the other file formats, these tests 
were run for a smaller set of data files. The results shown in 
Figure 3 verified the expectation of slower times for the Excel 
and XML format. 


Rtaiitng Data 
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Figure 2: Comparing the data formats read into SPSS Modeler. The CSV 
and SPSS .sav files were stored locally on the server system and the DB2 
database was running on a remote system. 


Figure 3\ Comparing the data formats read into SPSS Modeler. Excel and 
XML were the slower performing data formats. 
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Sorting data 

The sorting test involved sorting the data sets by three 
columns, using the stream shown in Figure 4. In order to 
provide a more realistic reflection of a customer scenario, 
the times include how long it took to read the data as well 
as the time taken to sort. 


Figure 5 shows that SPSS Modeler’s sorting performance 
performs in a linear manner as the number of records sorted 
is increased. The test results show that the use of SQL 
pushback functionality provides a signiflcant increase in 
performance for the sorting operation. The increase in 
performance is because by enabling SQL pushback within a 
stream, the SQL instructions are pushed back and executed 
in the database itself 



TextFile Sort SampleNoRecords SortData 


Figure 4: Test stream for sorting data with SPSS Modeler. 


Reading and Writing Data 



Test Case 

CSV 

DB2 Table 

-DB2 Table (SQL pushback) 


Figure 5\ Sorting data with SPSS Modeler. SQL pushback improves the 
performance when using a database. The CSV file was stored locally on the 
server system and the DB2 database was running on a remote system. 
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Data aggregation 

A five million record data set was used to test aggregating 
with SPSS Modeler by scaling the number of unique values 
appearing in the designated field using the stream shown 
in Figure 6. In order for the test results to truly reflect how 
it would be used in an operating environment, the times 
measured for the SQL pushback also include the time it 
takes to read in the data. 


The test results show that the aggregation operation scales 
well as the number of unique categories to be aggregated 
increases. Figure 7 highlights the performance times which 
again highlight the drastic improvement in SPSS Modeler’s 
SQL pushback functionality when performing aggregation 
using a database table as the source data compared to a CSV 
file. With SQL pushback the process was complete in almost 
half the time of the CSV file. 



TextFile Aggregate SampleNoRecords AggregateCSV 


Figure 6'. Test stream to aggregate data with SPSS Modeler. 


Data Aggregation 



Data Source 

CSV 

— DB2 Table (SQL pushback) 


Unique Humber of Categories to Aggregate 
(millions) 


Figure 7: Aggregating data with SPSS Modeier. Using SQL pushback within 
DB2 was aimost twice as fast as a CSV File. 
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Merging data 

To check the merge operation’s performance, times were 
recorded as the size of the data sets increased, using the stream 
in Figure 8. An inner join was performed using a unique “ID” 
column, meaning that the merge was a one-to-one merge. 

In other words, every record in the first data set had only 
one match in the second data set. 


The test results show that the merge operation scales relatively 
linearly in relation to the number of records being merged in 
Figure 9. Yet again, the improvement in time with the use of a 
database and SQL pushback is evident. This is because when a 
merge is performed using SQL pushback, the merge has 
already taken place before the data is read out of the database 
and then brought into SPSS Modeler. Therefore, the times in 
this section include the time taken for the data to be read into 
SPSS Modeler. 



Figure 8'. Test stream to merge data with SPSS Modeler. 


Data Merging: Inner join 1-1 Relationship 



Data Source 

CSV 

DB2 Table (SQL pushback) 


Figure 9\ Merging data with SPSS Modeler. When a merge is performed 
using SQL pushback, the merge has already taken place before the data is 
read out of the database into SPSS Modeler. 
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Writing data 

Next, we measured the writing of data out to a CSV file, 
SPSS .sav file. Excel file, XML file and DB2 database tables 
using the stream shown in Figure 10. 


Figure 11 illustrates the linear performance as the data sets are 
scaled up for CSV files, SPSS .sav and database tables. The 
CSV and SPSS .sav files are stored locally on the server system 
and the DB2 database is running on a remote system. Please 
note that the “Via ODBC” setting was used when writing data 
to the database. 



Figure 10: Test stream to read and write data with SPSS Modeler. 


When executing streams of this nature, it is not possible to 
separate the data writing performance from the data reading 
performance because SPSS Modeler performs the reading and 
writing operations at the same time. In other words, before 
all the data has been read in, the data writing operation has 
already begun. Consequently, the times shown include both 
the reading and the writing of data. 


Reading and Writing Data 



File Format 

CSV 

DB2 Table 
-SPSS .sav 


Figure 11: Reading and Writing data with SPSS Modeler. As the read and write 
operations occur simultaneously, the time includes both. 
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To keep consistency with the read testing, the performance of 
Excel and XML file formats run on a smaller set of data as 
shown in Figure 12. The recorded times for these tests show 
that the slowest performing file format is the Excel format, 
which is not surprising given that the elapsed times include the 
reading of the file as well as the writing. It was also the slowest 
in the read test. 
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Model building 

Figure 13 shows an example stream that was used to conduct 
the testing for the model building execution in SPSS Modeler. 



TextFile Filter PredictFlag ModelBuild 


Figure 13'. Test stream used for model building - in this case a C5.0 model. 


The dataset used contained up to 500,000 records and each 
model was built independently. However to allow for easier 
comparisons of the models, the results have been put together 
into four groups: 

• classification models 

• segmentation models 

• association models 

• automated models 


Figure 12: Reading and writing data with SPSS Modeler. Excel remained the 
slower performing dafa formaf. The fimes include both read and write 
operations. 
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Classification models use the values of one or more input 
fields to predict the value of one or more output or target 
fields (for example, logistic regression or a decision tree). 

The majority of the models included in this group (Figure 14) 
completed building in less than 30 seconds. The slowest model 
to build was the Neural Net, which does take a longer amount 
of time due to the sophistication and learning that is required 
by the technique in building of the model. 


Segmentation models divide the data into segments or 
clusters of records that have similar patterns or characteristics, 
such as KMeans clustering or anomaly detection. Although 
the KNN could be included with the classification models, 
its performance is closer to those in the segmentation group, 
due to the nature of classifying cases based on similarity to 
other cases nearby. The Two Step, KMeans and Anomaly were 
the quicker of the models and finished building around at 29, 
47 and 66 seconds respectively on the 500,000 data set. 


Model Building: Clatslflcatlon Models 
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Model Kiiildinq: SngtiinntiiliGii Models 
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Figure 14: Model building times for classification models in SPSS Modeler. 

Most of the models completed in less than 30 seconds (0.5 minutes). Figure 15: Model building times for segmentation models in SPSS Modeler. 

The Two Step, KMeans and Anomaly were the quickest of the five models. 
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Association models are used to find patterns data where one 
or more entities (such as events, purchases, or attributes) are 
associated with one or more other entities to construct rule 
sets that define these relationships. Figure 16 shows that both 
the Carma and Apriori models were built in less than 15 
seconds on the 500k data set (0.25 minutes). 


Model Building; Anoclallen Models 



Modi-I 

acngti 

carma 


Figure 16: Model building times for association modeis in SPSS Modeier. 
Apriori completed in less than 15 seconds while Carma was less than 10. 


The automated models (Auto Classifier, Auto Cluster and 
Auto Numeric) estimate and compare a number of different 
modeling methods, allowing a variety of approaches to be 
attempted in a single modeling run. Although ALM 
(Automated Linear Model) does not use multiple algorithms 
to build and evaluate, it does contain an automated data 
preparation step which internally transforms the target and 
predictor variables in order to maximize the predictive power 
of the model. Figure 17 shows the three automated models 
are relatively linear in performance —taking less than eight 
minutes to build a range of the models on the 500,000 data 
set and ranking them based on performance. ALM completed 
its regression model in approximately two minutes. 


Mod«l Duilding; Automittd Modtis 
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Figure 1 7: Model building times for the automated models in SPSS Modeler. 
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Model scoring 

Scoring happens when you take a model and apply it to data to 
generate a prediction or score. It can be done in batch —for 
instance, scoring customers each month whose contract is up 
for renewal next month on their likelihood to cancel. Or it can 
be done in real time —such as a customer that is on the phone 
to a call center agent as their risk of cancelling is assessed at 
that point in time. The model scoring performed in this testing 
was batch scoring, with the models used for model scoring 
built using a data set with 10,000 rows and 20 columns. The 
resulting model was then used in a stream shown in Figure 18 
and files of various sizes were then scored. 



TextFile C5 SampleNoRecords ModelScore 


Figure 18: Test stream used for model scoring - in this case a C5.0 model. 


The following results show that as the number of records 
being scored increases, the performance of many models 
increases to a point and then remains constant. This increase 
is related to the fact that there is an initial fixed overhead 
related to the scoring process. This overhead is not related 
to the number of rows scored, rather a one-off cost. 
Therefore this becomes less important as the number of 
rows to be scored increases. 


The following three figures illustrate the performance of 
the scoring models for classification models (Figure 19), 
segmentation and association models (Figure 20) and 
automated models (Figure 21). Note that the charts show 
the scores per second rather than elapsed time to allow for 
better comparisons between the test cases and your actual 
data scoring requirement. 


Model Scoiing: Clmnilicolion Modeh 
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Figure 19-. Model scoring times for classification models In SPSS Modeler. 
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Model Scoring; Sogmonlation 4 Aseocietion Mod# It 
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Figure 20: Model scoring times for segmentation models in SPSS Modeler. 


Figure 21 : Model building times for the automated models in SPSS Modeler. 
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Optimizing performance 

Most of the SPSS Modeler Server’s high performance is 
achieved through performance optimizations that are switched 
on by default. There will be times, however, when analysts and 
data miners need more control over the optimization of their 
SPSS Modeler streams. SPSS Modeler Server supports this 
by providing immediate feedback upon execution. 

In Figure 22, the SPSS Modeler stream is executed using SQL 
generation and the nodes turn purple rather than the usual 
white. Purple nodes indicate the operations represented by 
those nodes have been translated into SQL and executed 
in-database. This feedback helps ensure that as much of the 
stream as possible is executed in the database. Additional 
options allow the user to examine the SQL that is generated. 



SQL pushback to improve model scoring times 

Certain models within SPSS Modeler Server have functionality 
that allows the SQL to be generated, pushing back the model 
scoring stage to the database. This allows streams containing 
these models to have their full SQL pushed back to the 
database. 

The models that support this functionality are: 

• C5.0 

• Classification & Regression Tree (CART) 

• CHAID 

• Quest 

• Decision List 

• Logistic Regression 

• Neural Net 

• PCA 

• Linear Regression 


Figure 22: SQL generation and highlighting in a SPSS Modeler stream. The 
nodes have turned purple to indicate those nodes have been transiated into 
SQL and executed in-database. 
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Figure 2 3 confirms that the model scores per second rapidly 
increase by enabling SQL pushback. The Logistic regression 
model experienced a gain of more than 47 percent. 


Model Scoring: SQL pushback 



List 

Model 


pushback 

□ off 
■ on 


Figure 23'. Model scores per second with and without SQL pushback. SQL 
pushback is a feature only available In the server version of IBM SPSS 
Modeler. 


In-database mining to reduce data movement 

Many organizations have invested heavily in a database 
infrastructure for predictive analytics and business intelligence 
systems, but these systems are often under-utilized by the 
analytical tools that use them. One of the key benefits of 
SPSS Modeler Server is that it allows organizations to fully 
utilize their investments in high-performance database systems 
by taking advantage of database optimization and reducing 
data movement. 

The following vendor algorithms are available in SPSS 
Modeler for use with the respective database: 

• IBM InfoSphere: Association, Clustering, Decision Tree, 
Logistic Regression, Naive Bayes, Regression, Sequence, 
Time Series. 

• IBM Netezza: Bayes Net, Decision Trees, Divisive 
Clustering, Generalized Linear, K-Means, KNN, 

Linear Regression, Naive Bayes, PCA, Regression Tree, 
Time Series. 

• Microsoft SQL Server: Association Rules, Clustering, 
Decision Tree, Linear Regression, Naive Bayes, Neural 
Network, Sequence Clustering, Time-Series. 

• Qracle: Adaptive Bayes, Apriori, Artificial Intelligence (AI), 
Decision Tree, General Linear Model (GLM), KMeans, 
Minimum Description Length (MDL), Naive Bayes, 
Non-Negative Matrix Factorization, Q-Cluster (Qrthogonal 
Partitioning Clustering), Support Vector Machine. 
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SPSS Modeler Scoring Adapters 

By using SPSS Scoring Adapters, model scoring can be 
performed in-database to avoid the need to move the data out 
of the database. This is made possible by installing a set of 
User Defined Functions (UDFs) that perform the model 
scoring within the database. The Scoring Adapters include a 


very large number of the SPSS Modeler models, to allow for 
more scoring operations to be moved to the database than just 
those which can leverage SQL pushback. Scoring Adapters are 
currently supported for three database types: IBM Netezza, 
IBM DB2 for z/OS and Teradata. 



SQL pushback 

In-database 

support 

Scoring Adaptor 
support 

Read-Write 
(no SQL pushback) 

Read only 

(no SQL pushback) 

IBM DB2 Enterprise 
Server Edition 

X 

X 




IBM DB2fori 
(formerly i5/OS) 

X 





IBM DB2 for z/OS 

X 


X 



IBM Informix 




X 


IBM Infosphere Classic 
Federation Server for 

z/OS 





X 

IBM Netezza Data 

Warehouse 

X 

X 

X 



Greenplum Database 

X 





Microsoft SOL Server 

X 

X 




MySQL 




X 


Oracle Database 

X 

X 




Salesforce.com 





X 

SAP Hana 

X 





SAP Sybase IQ 

X 





Teradata 

X 


X 




Table 1 : A high level summary of the in-database capabilities of supported databases. 
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Intelligent SQL generation within stream 
execution to improve performance 

SPSS Modeler Server intelligently re-orders operations in 
the SPSS Modeler stream to maximize performance without 
altering results. Analysts or data miners can organize streams 
in a way that makes sense to them, while SPSS Modeler 
Server reorganizes those operations in a way that makes sense 
to the database. Figure 24 contains a “Derive” node which 
contains an operation that cannot be carried out in the 
database, whereas the “Select” node can be pushed back to 
the database, as indicated by its purple color. 


Normally, the cache is stored as a temporary file on the file 
system, but SPSS Modeler Server also supports the caching 
of this data into a temporary table in the database. When 
combined with SQL optimization, this may result in significant 
gains in performance. By automatically generating SQL for 
all downstream nodes, performance can be improved further. 

In Figure 25, the “Merge” operation is highlighted, indicating 
that the operation is being executed in the database from the 
filled database cache. 



Data Input Derive Select Table 



OMa «ciul 




Figure 25'. Setting a cache on a node that is iikeiy to be re-executed wiii store 
the data in a temporary tabie on the database (where possibie), aiiowing 
further in-database operations from that node on. 

In-database caching 

Qne common user optimization is to set up a cache on a 
node. The next time data is passed through that node, the 
cache is filled with that data. From then on, the data is read 
from the cache rather than from the data source. This can 
be a useful way to ensure that expensive data processing is 
only executed once. 


Figure 24'. SPSS Modeier Server optimizes the process so that the “Seiect” 
operation is performed before the “Derive” operation, reducing data transfer 
and improving performance. 
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Optimizing for very large data sets 

SPSS Modeler Server features options associated with the 
selected models, allowing the users to specify that they are 
working with very large data sets (VLD—referred to as 
“PSM” options during benchmark testing). Essentially these 
work by dividing the data into smaller data sets and building 
one model on each data set. The most accurate models are 
then automatically selected and assembled to create a single 
final model nugget. The tests, outlined below, focused 
on the scalability of the VLD options and were tested 
for comparison with a Neural Net model. These tests 
demonstrated considerable time savings when working with 
VLD and leveraging SPSS Modeler Server’s VLD options 
on multi-processor machines. 

This test was nm building the Neural Net model using data 
ranging from one million records to ten million records.^ By 
taking advantage of a system’s additional CPUs and parallel 
processing, the PSM test used approx 20 GB of the systems 
large RAM capacity while increasing performance. 


1M 



s 

J » 
t 


0 


PSM 

TM«Typ« 
—PSM |I CPU») 


2 



Figure 26\ SPSS Modeler PSM functionality can take advantage of a 
system’s additional CPUs in order to use parallelism to increase the 
performance of the Model building functionality. 


Advanced performance optimization 

Aside from in-database mining, caching, modeling and scoring, 
SPSS Modeler and SPSS Modeler Server provide a number of 
additional capabilities that enable data miners to optimize the 
performance of their streams. 
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Database bulk-loading to relieve bottlenecking 

Data movement is often a bottleneck in performance— 
especially when writing to a database. SPSS Modeler provides 
a number of features to optimize this process for large data 
volumes. By default, writing to a database is performed on 
a row-by-row basis. This prevents errors and provides data 
security but slows performance. Allowing SPSS Modeler 
to commit multiple rows at a time is a good way to ensure 
more reasonable performance —and this option is available 
by default. 

In addition to the batch committal of records, SPSS 
Modeler supports two types of bulk loading. One is provided 
through ODBC bulk loading facilities and the other uses an 
external bulk loading tool to allow a database-native solution 
(Figure 27). 



Figure 21-. The DB Export: Advanced Options dialog box easily enables 
bulk loading to the database via ODBC or through an external loader. 
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External bulk loading scripts are provided for IBM DB2® 
Intelligent Adiner® for Data, IBM Netezza Performance 
Server, IBM Redbrick® Warehouse, Microsoft SQL Server, 
Oracle Data Miner and Teradata Warehouse databases. These 
scripts can be customized and custom scripts may be written 
for other databases. 

Database indexing 

Indexing database tables maintains the performance of 
in-database options. Correct indexing significantly impacts 
many subsequent database operations. 

SPSS Modeler Server enables users to create indices on tables 
exported from SPSS Modeler (Figure 28). Simple indices can 
be created easily. SPSS Modeler also allows users to customize 
the SQL statement used to create the index (for instance, to 
create a BITMAP, UNIQUE, or FILLFACTQR index). 



Optimized joins and sorts 

By default, SPSS Modeler has to make assumptions about the 
state of data in the system. For example, SPSS Modeler cannot 
assume that any data has already been sorted. Therefore, many 
operations ensure a sort is performed even if such a sort is 
redundant. SPSS Modeler allows the user to optimize a “sort” 
or “join” operation by specifying any existing sorts on the data. 
This eliminates redundancy and improves performance. 

Users can also optimize the performance of SPSS Modeler 
through special case algorithms for joins. SPSS Modeler’s 
default join algorithm is designed to perform optimally when 
joining data sets of similar size. 

In some very common operations, such as when using a join 
to connect an ID in one table to a label or description from 
another (for example, joining a product code in a table of 
transactions to a product name in a look-up table), the default 
join is inefficient. SPSS Modeler offers an alternate join 
algorithm for these situations which significantly boosts 
performance speed. 


Figure 28: Create indices on database tabies from within iBM SPSS Modeler 
Server to improve database performance. 
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Parallel processing to improve performance 

Symmetric Multi-Processor (SMP) machines are widely used 
and available for all platforms supported by SPSS Modeler 
Server. They comprise multiple CPUs sharing access to the 
same memory, disk, network and other I/O resources. When 
a multi-threaded application runs on an SMP box, threads 
may be distributed across the CPUs and execute truly in 
parallel. Application processes and individual threads can 
usually migrate dynamically between CPUs to balance 
processor load. This is generally handled transparently by 
the operating system. 

SPSS Modeler uses a parallel data sorting algorithm to 
improve the performance of a number of data processing 
operations. Sorting is used by many SPSS Modeler operations 
including binning, model evaluation, merge and the sort 
operation itself. All of these operations benefit from the 
parallelization of the sort operation. 

The parallelized sort algorithm uses a technique called 
“record parallelism.” This technique assigns records on a 
round-robin basis to separate sorting processes. Each process 
sorts its own subset of records and the results are joined. Sort 
times can be reduced by more than 30 percent when running 
on multi-processor hardware and at high data volumes. 


Scoping and sizing SPSS Modeler Server 

Many factors must be considered when scoping hardware 
requirements for an SPSS Modeler Server installation. 

The breadth of operations and differences in data volumes 
make it difficult to estimate performance for any specific 
hardware configuration. 

Impact of CPUs on performance 

Obviously the core speed of any individual CPU will impact 
data mining performance. Almost all data mining operations - 
especially modeling —are heavily processor dependent, so an 
increase in CPU speed will produce a proportional increase 
in performance for many SPSS Modeler processes. 

The main benefits of multiple CPUs (or multicore CPUs) 
occur when running multiple streams. This means that 
the number of users will often be the deciding factor in 
determining the optimum number of CPUs. Multiple CPUs 
will also benefit parallelized operations, but the main benefits 
will be from supporting multiple users (Table 2). 


Number of users 

Number of CPUs 

1-2 

1 

3-4 

2 

5-10 

4 

11-20 

8 

21+ 

16 


Table 2: CPU recommendations for Modeler Server based on the number 
of users. 
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For a production server running scheduled data mining via 
IBM® SPSS® Collaboration and Deployment Services, the 
number of CPUs should be determined by the number of 
separate processes to be performed simultaneously. Maximum 
performance can be achieved, for instance, by splitting a 
model scoring process across multiple CPUs or building 
multiple models simultaneously. 

Impact of physical memory on performance 

Most SPSS Modeler operations can be performed on large 
volumes of data with minimal memory usage. Only certain 
operations, such as sorting, joining, and modeling, require 
data to be temporarily stored in memory. If not enough 
memory is available, these operations will store part of the 
data as virtual memory on disk. This can affect performance, 
since disk access is significantly slower than memory access. 

As with CPU usage, the number of users impacts the required 
memory for normal operation. Memory requirements depend 
on data volume. 


Impact of disk space on performance 

Before addressing disk space requirements, it is important to 
understand the volume of data that is likely to be used for the 
actual data mining. Most organizations store many terabytes 
of data —especially transactional data —but this amount 
will rarely be used. Normally the data is aggregated, selected 
or sampled before it is used for analysis. While large data 
volumes are typically used in model scoring, the model scoring 
processes usually rely on operations that do not use a lot of 
system resources. 

When trying to maximize performance, disk usage for data 
processing steps can be relatively high. The user often caches 
data to minimize execution times, and some operations will 
“spill to disk” when physical memory is unavailable. In 
addition, some operations may produce a dataset larger than 
the raw input data, further increasing disk requirements. 

Given that the large data preparation steps are typically 
executed infrequently (it is best practice to store the results 
of such processing as intermediate files or tables), a 
conservative rule of thumb is to reserve between three to 
five times the disk space required to store the original data. 
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Conclusion 

SPSS Modeler Server’s combination of high performance, 
scalability, performance optimizations and flexible 
hardware requirements enable it to handle large and 
complex data mining projects. With SPSS Modeler Server, 
organizations can: 

• Maximize their investment in high performance databases 
for all data mining tasks —ensuring high performance and 
minimizing data transfer costs. 

• Leverage SQL pushback and Scoring Adapters generation 
to improve performance. 

• Use in-database caching, database write-back with indexing 
and optimized merging to join tables outside of the database. 

• Incorporate data mining algorithms from other database 
vendors. 

• Maximize use of multiple CPUs (or multi-core CPUs) in an 
operating environment by using parallel processing during a 
number of data preparation and model-building operations. 

The ever-growing amount of data created by organizations 
presents opportunities and challenges for data mining. Scaling 
the entire data mining process with SPSS Modeler Server 
makes it possible for your organization to analyze large 
volumes of data effldently—reducing the time needed to 
turn data into better business decisions that boost your ROI. 


About IBM Business Analytics 

IBM Business Analytics software delivers data-driven 
insights that help organizations work smarter and 
outperform their peers. This comprehensive portfolio 
includes solutions for business intelligence, predictive 
analytics and decision management, performance management, 
and risk management. 

Business Analytics solutions enable companies to identify 
and visualize trends and patterns in areas, such as customer 
analytics, that can have a profound effect on business 
performance. They can compare scenarios, anticipate potential 
threats and opportunities, better plan, budget and forecast 
resources, balance risks against expected returns and work to 
meet regulatory requirements. By making analytics widely 
available, organizations can align tactical and strategic 
decision-making to achieve business goals. For further 
information please visit ibm.com/business-analytics. 

Request a call 

To request a call or to ask a question, go to 
ibm.com/business-analytics/contactus. An IBM 
representative will respond to your enquiry within 
two business days. 
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Agent-Based Social Simulation in the Service of Child Welfare Issues: 
Defining the Demand for Group and Residential Care 


Dr. Fred H. Wulczyn, Chapin Hall Center for Children at the University of Chieago 
Dr. John R. Hummel, Advaneed Simulation Teehnologies Center, ANL/DIS 

Introduction 

Chapin Hall researchers have a long track record of bringing advanced analytical techniques to 
bear on issues affecting children, ranging from clinical research using randomized trials to 
system dynamics modeling of process flows to observations of complex system behavior using 
chaos theory to search the high-dimensional state space for hidden structures in the data. Chapin 
Hall’s keen appreciation of the U.S. child welfare apparatus as an emphatically complex system 
led them, in the fall of 2006, to explore the potential benefits of collaboration with ASTC to see 
if technologies such as agent-based simulation might help unlock some of the mysteries of their 
problem domain. Argonne’s ASTC has, for well over a decade, been an incubator for advances 
in fine-scale, multidisciplinary simulation of heterogeneous, complex systems, including, for 
example, clinical, physiological and logistical simulations of healthcare delivery, adaptive social 
dynamics of the South American cocaine trade, and studies of the socioecological sustainability 
and social stability of settlement systems under stress, both in ancient (Mesopotamia) and 
modem (Southeast Asia) milieus. 

This work has benefitted directly from ASTC’s ongoing agent-based simulation R&D, 
particularly with regard to ASTC’s work, in collaboration with the RAND Corporation, on 
simulation of potential impacts of national healthcare reform proposals, and in its NSF- 
sponsored holistic socioecological sustainability studies, being conducted in collaboration with 
the Oriental Institute of the University of Chicago. 

Problem Statement 

Each year, state child welfare agencies investigate roughly 3 million reports of child 
maltreatment. Of those reports, nearly one million are substantiated. Treating abused and 
neglected children costs federal, state, and local governments about $20 billion dollars each year, 
although that figure does not include the added costs of health care, mental health services, or 
specialized educational services. Given that one-half of the children entering out-of-care have 
are behind their peers on a wide range of developmental indicators, the tme costs of caring for 
abused and neglected children are much, much higher. 

The services available to abused and neglected children are an admixture of in-home and out-of¬ 
home services, if services are in fact available. For the most part, maltreated children are not 
referred for services at all. When children are referred for services, in-home services include 
parenting classes, homemaker services, counseling, and other supports designed to improve the 
parents’ ability to raise their children safely. When the safety risks to the child are too great, 
public officials resort to out-of-home care, a course of action that involves locating another home 
for the child, at least temporarily. Out-of-home care, otherwise known as foster care, ranges 
from family-based foster care, relative/kinship foster care, group homes, and residential 


1 



institutions.' Placement oecurs in about 25 percent of the eases that are substantiated, a figure 
that translates into roughly 250,000 admissions annually. Of all the children admitted to foster 
care, infants and teenagers are the children at greatest risk. Among infants, plaeement rates per 
1000 ehildren of the same age in the general population reach above 5 pereent in the poorest 
areas of the country. All in all, about 1 in 5 children placed in foster care is placed before his or 
her first birthday. Placement rates per 1000 among teens are lower; nevertheless adolescents 
(children between the ages of 13 and 17 at admission) account for between 35-40 pereent of the 
children who enter the plaeement system. 

Notwithstanding the large number of children who are left without any service at all, publie 
policy favors in-home services so that a child’s family life is not disrupted. When that is not 
possible, public policy favors placement of children in the most family-like setting, given the 
needs of the child. That said, for some age groups of children, group and residential care aecount 
for more than half of all placements into out-of-home care. In some jurisdictions, the likelihood 
a teenager will be placed in a group or residential setting goes as high as 80 percent. Even 
among infants, plaeement in a group setting is surprisingly high. Reeent researeh carried out at 
the University of Chieago suggests that group care utilization rates for infants range between 3 
and 35 percent. Although we generally think of orphanages as Dickensian relics, the truth is, 
vestiges of the old way of caring for children are with us still. 

What is perhaps most striking about the plaeement system in the U.S. is how little we use what 
little we know to pursue better services for what is arguably the nation’s most vulnerable group 
of children. For younger ehildren, partieularly those under the age of 1, there is virtually no 
developmental research that supports the use of group care as an alternative to a single, primary 
set of caregivers.^ Even placement with strangers in a family setting has signifieant iatrogenie 
effects when very young ehildren are involved. Among adolescents, the researeh is more 
equivocal although it leans decidedly toward redueing our reliance on group settings for 
adolescents who eannot live at home. 

Despite the evidence, the praetice of plaeing children into group and residential eare persists. 
The question is why and what ean be done to align practice with what is known about the 
benefits of group care? In our view, the praetiee of placing children into group settings lingers 
beeause the supply of beds dietates the demand.^ The supply lingers beeause public officials are 


' Group homes usually house between 4 and 12 unrelated children, although the exact capacity is 
regulated by states. Residential care settings are much larger, with some residential care programs having 
a capacity for several hundred children who live in cohages on a campus. 

^ To be fair, no state or provider organization advocates raising very young children in a group setting. 
Placement in shelter care (a form of group care) is thought of as a way to manage demand for placement 
that oscillates over relatively short cycles. Nevertheless, the supply of shelter beds has become, we 
believe, its own source of demand. 

^ One example of how demand for residential/group care is governed by the supply comes from New 
York City. During the latter half of the 1990s and early this decade, the overall number of foster children 
dropped from slightly less than 50,000 to 25,000 because of a steep drop in the number of admissions. 
Over that same time period, the population of children living in group care remained virtually constant 
because the supply of group care beds remained largely unchanged. In the face of declining demand, the 
persistent level of group care utilization implies micro-level decision processes that are susceptible to the 
influence of supply. 
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simply unable to describe the long run demand for group and residential care other than to assert 
that the demand is not zero. Given the significant bricks and mortar investments residential care 
represents, providers are unwilling to manage their supply of services up or down without a clear 
indication of where the market for their services is headed. If the optimal quantity of group and 
residential care is greater than zero, but less than the current supply (as current research implies), 
any given provider may simply be waiting to see which one of their competitors downsizes (or 
goes out of business altogether). In the meantime, use of their beds remains relatively stable as 
utilization seeks its equilibrium around capacity. 

Technical Approach 

In technical terms, we see residential/group care as a complex system that exhibits adaptive 
behavior consistent with the presence of multi-loop feedback mechanisms. The system operates 
at both the macro-level of governments and other organizations as aggregate, decision-making 
agents and at the micro level of individual persons as cognitive agents. 

The dynamics of the residential/group care issues articulated above play out in the context of a 
heterogeneous, multidisciplinary complex system: the overall child welfare apparatus. Figure 1 
provides a simplified sketch view of the system components relevant to the group care pathway 
and its various alternatives. The figure depicts flows of individual person agents representing 
needy children into and out of the various care alternatives. With data available through Chapin 
Hall, the flows, and the actions undertaken by, and upon, the individual person agents can be 
resolved on essentially a daily basis. In broad terms, children are admitted to a care pathway, 
receive care, and are released or removed from care, at which point they may not require further 
care and may leave the system, or may require care and reenter the needy population. The 
activities that take place in the course of this process are, of course, far more complex than could 
be shown in the figure. 

Currently we have developed a model (Ospedale) that utilizes data derived from administrative 
data sets to parameterize the agent based simulation model. The model can be adjusted by 
making different assumptions about these parameters. In addition the model can be tuned to 
operate with subpopulations and those can be simulated as subpopulations or in the aggregate 
with other subpopulations. There are three distinct categories of dynamic simulation models that 
are now built and operate in the Ospedale modeling environment. 

1. Models of decisions to admit children in need of care to one of the care path alternatives, 
based upon historical data and corresponding statistical models of admissions. 

2. Models of changes in licensed system capacity (i.e., number of beds) of facilities due to 
endogenous and exogenous factors, again based on historical data and statistical models. 

3. Models of clinical outcomes, including duration of stay, as a function of individual 
patient characteristics, demographics, family background, medical history, etc., and the 
profile of care that the child was given. 

In addition plans are in place and to build additional functionality in order to reasonably portray 
the dynamics of the system and include: 
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1. Models of monetary eost, resources consumed, and assets tied up, in providing various 
increments of care, to allow tracking of system performance under a load and to allow 
integration of total costs, based upon historical data and corresponding statistical models. 


NOTE: Residential Care and 
Other Alternative Care Paths 
have complexity (not shown) 
comparable to Group Care 


TYPE 2 SIM MODELS 
Licensed Capacity: 
f( contract 
requirements, 
govt, policies, etc.) 


TYPE 3 SIM MODELS 
Clinical Care Path: 

f( patient condition, 
resources available, 
clinical guidelines ) 


TYPE 4 SIM MODELS 
Cost of Care: 

f( patient acuity, 
resources used, 
duration) 


Figure 1: Simulated Dynamics of Care for Capacity Planning 


TYPE 6 SIM MODELS 

other non-clinical 
social mechanisms 
may come into play 
here and elsewhere. 



Successes: 
children leaving 
the Needy 
Population 


2. Summary data on efficacy of the healthcare options exercised in patient care paths can be 
used to help build these models. 

3. Diverse models of other societal processes that may trigger and modulate the response of 
the child support system, including, for example, interactions with law enforcement, 
family dynamics (changes in employment status of breadwinners, etc.) and so on. 


We continue to build and test this multifaceted simulation system using relatively simple 
simulation formulations and gradually add complexity and fidelity. We have structured the 
simulation such that it will be simple to swap alternative models of the various processes in and 
out of the overall framework, to allow us to try out various alternative hypotheses regarding 
these complex processes, and see which ones tend to produce the most realistic and reasonable 
outcomes. This will, of course, make the system far more useful and accessible for other 
researchers who may wish to use the simulation system in future. 


Data Resources 

For the past fifteen years, researchers at Chapin Hall Center for Children, through the Center for 
State Foster Care and Adoption Data have been amassing the nation’s most extensive collection 
of placement data. Extracted from the administrative records of public children welfare 
agencies, the data base tracks the full placement history for 1.7 million children in 20 states 
representing one-third of the nation’s counties. In addition, the data have been linked to 1990 
and 2000 census data, making it the only source of data for studying temporal and spatial 
dynamics. Of particular importance, the data support analysis of population dynamics (i.e., 
modeling admission/discharge dynamics as birth/death processes) at a variety of temporal scales, 
including daily changes. Finally, the placement data for Illinois has been linked at the child level 
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to a variety of other administrative data sourees, ineluding sehool reeords, Medieaid reeords, and 
ineome maintenanee data, to name a few other resourees. 

Most existing researeh with administrative data focuses on individual (i.e., person-level) 
experiences/processes. We believe that a socioecological orientation to the problem of 
extracting knowledge about human built systems ought to be applied. Doing so successfully 
through simulations methods broadens the range of applicable theory and extends dramatically 
the range of questions that can be answered with administrative databases. At the same time 
there is no question that the proposed work constitutes substantial new challenge for high- 
performance computing, both in mining and sifting the enormous transactional databases needed 
to extract social behavior pattern vignettes for simulation and in the fine-scale, multi-year 
simulations of the diverse activities of hundreds of thousands of individual children and 
caregivers in our agent-based simulations. 


Proposed Budget 

The project will avail itself of computing resources already on hand at the collaborating 
institutions, including ASTC’s 16-processor AMD Opteron mini-cluster computer, and very 
likely Argonne’s Blue Gene supercluster computer as well, so that no additional equipment 
acquisitions will be required. All project costs will be used for graduate and undergraduate 
student labor, as detailed below. The matching contribution from Chapin Hall will fund costs 
associated with bringing into the modeling building exercise experts with subject matter 
expertise and to support the involvement of senior level researchers as needed. 
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Chapin Hall 
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Pierson Computing Connection, Inc. 
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ChapinHaU 


al 1 he University oldliicago 

I’oliey research that heiielUsehildren. fainilies,and theireoninuiiiities 


April 16, 2014 


Dr. John D. Fluke 

Associate Director for Systems Research and Evaluation 

Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

The Gary Pavillion at The Children’s Hospital, Anschutz Medical Campus 

13123 E. 16* Avenue, B390 

Aurora, CO, 80045 


Dear Mr. Eluke: 

I am writing to confirm our intention to participate in your proposed work to design and 
implement a decision support tool that uses predictive analytics to identify service needs for 
foster children and the elderly. 

Should you be able to successfully obtain a contract from the Allegheny County Department 
of Human Services for this work, we intend to subcontract with you to provide services 
related to Integrated Outcome Management and Assessment and predictive modeling. 

We are prepared to provide appropriate staff and consultants to meet the needs of the project. 
The proposed work represents a terrific opportunity to apply our expertise in simulation 
modeling and predictive analytics, and we are looking forward to an opportunity to work 
with you and the staff at the Kempe Center. 

We look forward to learning whether our proposal was successful. 


Sincerely, 



1,A.^ 


Ered Wulczyn 
Senior Research Eellow 


131.'! East 60th Street T; 773.256.5100 
Chicago, IL 60637 F: 773.753.5940 


\w\v.ehapinhall.org 


Intelligent Outcome Analytics, LLC 


David R. Schwartz 
Intelligent Outcome Analytics, LLC 
17 Llanfair Road, Suite 1 
Ardmore, Pennsylvania 19003 


April 17, 2014 


Dr. John Fluke 

University of Colorado Denver 
Department of Pediatrics 
Kempe Center 
13123 E 16thAve.,B390 
Aurora, CO 80045 


Dear Dr. Fluke: 

I am pleased to support your proposal to provide a framework for predictive 
analytic modeling and predictive modeling services to the Allegheny County 
Department of Human Services. 

I look forward to leading the predictive analytic modeling component of this 
important project, serving as CO-PI, and collaborating with the project team 
members and Allegheny County administration and staff 

This is an exciting opportunity to collaborate with internationally respected 
scholars on a project that successfully aligns the goals/research of the collaborators 
with the goals of the overall project. 

I look forward to collaborating with you on this work. 

Best regards, 

David R. Schwartz 

CEO, Intelligent Outcome Analytics 
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Dr. John D. Fluke 

Associate Director for Systems Research and Evaluation 

Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

The Gary Pavillion at The Children's Hospital, Anschutz Medical Campus 

13123 E. le* Avenue, 8390 

Aurora, CO, 80045 


Dear Dr. Fluke, 

I am writing to confirm our intention to participate in your proposed work to design and implement a 
decision support tool that uses predictive analytics to identify service needs for foster children and the 
elderly. 

Should you be able to successfully obtain a contract from the Allegheny County Department of Human 
Services for this work, we intend to subcontract with you to provide software and services related to 
IBM SPSS Modeler and possibly other services as the plan for the project is developed. 

We are prepared to provide appropriate staff and consultants to meet the needs of the project. The 
proposed work represents a terrific opportunity to apply our expertise and services and we are looking 
forward to an opportunity to work with you and the staff at the Kempe Center. 

We look forward to learning whether our proposal was successful. 



Director of Sales 

Pierson Computing Connection, Inc. 


Pierson Computing Connection, Inc. 
10 Long Lane, Suite 100 
Mechanicsburg, PA 17050 








